Introduction to Deep Learning: With Complexe Python and TensorFlow Examples

Description

In Computer Sciences there is currently a gold rush mood due to a new field called "Deep Learning". But what is Deep Learning? This book is an introduction to Neural Networks and the most important Deep Learning model - the Convolutional Neural Network model including a description of tricks that can be used to train such models more quickly. We start with the biological role model: the Neuron. About 86.000.000.000 of these simple processing elements are in your brain! And they all work in parallel! We discuss how to model the operation of a biological neuron with technical neuron models and then consider the first simple single-layer network of technical neurons. We then introduce the Multi-Layer Perceptron (MLP) and the Convolutional Neural Network (CNN) model which uses the MLP at its end. At the end of the book we discuss promising new directions for the field of Deep Learning. A famous physicist once said: "What I cannot create, I do not understand". For this, the book is full of examples of how to program all models discussed in Python and TensorFlow - Today, the most important Deep Learning library. About the author: Prof. Dr.-Ing. Juergen Brauer is a professor for Sensor Data Processing and Programming at the University of Applied Sciences Kempten in Germany where he holds a "Deep Learning" and other machine learning related lectures for Computer Science and Advanced Driver Assistance Systems students. His personal experience tells him: "What I cannot program, I do not understand". "How are they called? Neutrons?"......Page 6 Convolutional Neural Networks drive the boom......Page 7 Neuroscience as a treasure for machine learning......Page 15 About this book......Page 18 Exponential growth of interest......Page 21 Acquisition of DL startups......Page 24 Hardware for DL......Page 27 Software for DL......Page 31 Your brain - A fascinating computing device......Page 33 Structure of a neuron......Page 36 Signal processing by action potentials......Page 38 Synapses......Page 40 Neuronal plasticity......Page 41 Spike-Timing Dependent Plasticity (STDP)......Page 43 What is the function of a biological neuron?......Page 47 Neurons as spatial feature or evidence detectors......Page 48 Neurons as temporal coincidence detectors......Page 52 Perceptron neuron model......Page 53 Neurons as filters......Page 56 Other neuron models......Page 61 Neural Coding......Page 62 The Perceptron neuro-computer......Page 65 Perceptron learning......Page 67 Perceptron in Python......Page 69 Limitations of the Perceptron......Page 77 The SOM neural network model......Page 83 A SOM in Python......Page 89 SOM and the Cortex......Page 101 The goal......Page 108 Basic idea is gradient descent......Page 109 Splitting the weight change formula into three parts......Page 111 Computing the first part......Page 112 Computing the third part......Page 113 Backpropagation pseudo code......Page 117 MLP in Python......Page 119 Visualization of decision boundaries......Page 134 The need for non-linear transfer functions......Page 138 Introduction......Page 141 Training a linear model with TensorFlow......Page 150 A MLP with TensorFlow......Page 152 Introduction......Page 160 Some history about the CNN model......Page 164 Convolutional and pooling layers in TensorFlow......Page 167 Parameters to be defined for a convolution layer......Page 173 How to compute the dimension of an output tensor......Page 178 A CNN in TensorFlow......Page 179 Fighting against vanishing gradients......Page 195 Momentum optimization......Page 197 Nesterov Momentum Optimization......Page 200 AdaGrad......Page 201 RMSProp......Page 202 Adam......Page 203 Comparison of optimizers......Page 204 Batch normalization......Page 207 Principle of attention......Page 210 Principle of lifelong learning......Page 211 Principle of embodiment......Page 212 Principle of prediction......Page 213 Cognitive architectures......Page 214 Ex. 1 - Preparing to work with Python......Page 217 Ex. 2 - Python syntax......Page 221 Ex. 3 - Understanding convolutions......Page 224 Ex. 4 - NumPy......Page 227 Ex. 5 - Perceptron......Page 232 Ex. 6 - Speech Recognition with a SOM......Page 234 Ex. 7 - MLP with feedfoward step......Page 235 Ex. 8 - Backpropagation......Page 236 Ex. 9 - A MLP with TensorFlow......Page 237 Ex. 10 - CNN Experiments......Page 238 Ex. 11 - CNN for word recognition using Keras......Page 239 Ex. 12 - Vanishing gradients problem......Page 240 Ex. 13 - Batch normalization in TensorFlow......Page 241