Deep Learning Essentials

Description

Cover......Page 1 Title Page......Page 2 Copyright and Credits......Page 3 Packt Upsell......Page 4 Contributors......Page 5 Table of Contents......Page 7 Preface......Page 14 Chapter 1: Why Deep Learning?......Page 20 What is AI and deep learning?......Page 21 The history and rise of deep learning......Page 23 Advantages over traditional shallow methods......Page 29 Impact of deep learning......Page 31 The motivation of deep architecture......Page 33 The neural viewpoint......Page 34 The representation viewpoint......Page 35 Distributed feature representation......Page 36 Hierarchical feature representation......Page 38 Applications......Page 39 Success stories......Page 40 Deep learning for business......Page 47 Future potential and challenges......Page 48 Summary......Page 50 Chapter 2: Getting Yourself Ready for Deep Learning......Page 51 Data representation......Page 52 Data operations......Page 53 Matrix properties......Page 54 Deep learning with GPU......Page 56 Deep learning hardware guide......Page 57 RAM size......Page 58 Cooling systems......Page 59 TensorFlow – a deep learning library......Page 60 MXNet......Page 61 Theano......Page 62 Keras......Page 63 Framework comparison......Page 64 Setup from scratch......Page 65 Setup using Docker......Page 69 Summary......Page 71 Chapter 3: Getting Started with Neural Networks......Page 72 Multilayer perceptrons......Page 73 Activation functions......Page 74 Tanh or hyperbolic tangent function......Page 76 Leaky ReLU and maxout......Page 77 Weight initialization......Page 78 Backpropagation......Page 79 Backpropagation......Page 80 Automatic differentiation......Page 81 Vanishing and exploding gradients......Page 82 Regularization......Page 83 Convolutional Neural Networks......Page 84 Convolution......Page 85 Pooling/subsampling......Page 87 Overall......Page 88 Restricted Boltzmann Machines......Page 89 Encoding and decoding......Page 90 Contrastive divergence (CD-k)......Page 93 Recurrent neural networks (RNN/LSTM)......Page 94 Backpropagation through time......Page 95 Vanishing gradient and LTSM......Page 96 Cells and gates in LTSM......Page 97 Practical examples......Page 98 Handwritten digits recognition......Page 99 Summary......Page 103 Origins of CNNs......Page 104 Convolutional Neural Networks ......Page 106 Data transformations......Page 108 Input preprocessing......Page 109 Data augmentation......Page 110 Network layers......Page 111 Convolution layer......Page 112 Pooling or subsampling layer......Page 113 Fully connected or dense layer......Page 114 Network initialization......Page 115 Regularization......Page 116 Loss functions......Page 118 Model visualization......Page 119 Handwritten digit classification example......Page 121 Fine-tuning CNNs......Page 124 Popular CNN architectures......Page 125 Visual Geometry Group......Page 126 ResNet......Page 127 Summary......Page 128 Traditional NLP......Page 129 Bag of words......Page 130 Weighting the terms tf-idf......Page 131 Motivation and distributed representation......Page 132 Word embeddings......Page 133 Idea of word embeddings......Page 134 Advantages of distributed representation......Page 136 Commonly used pre-trained word embeddings......Page 137 Basic idea of Word2Vec......Page 139 The word windows......Page 140 Generating training data......Page 141 Negative sampling......Page 142 Hierarchical softmax......Page 143 The input layer......Page 144 The loss function......Page 145 Continuous Bag-of-Words model......Page 146 Training a Word2Vec using TensorFlow......Page 147 Using the pre-trained Word2Vec embeddings......Page 152 Understanding GloVe......Page 153 FastText......Page 154 Summary......Page 155 Chapter 6: Advanced Natural Language Processing......Page 156 Limitations of neural networks......Page 157 Recurrent neural networks ......Page 159 RNN architectures......Page 160 Basic RNN model......Page 161 Training RNN is tough......Page 162 Long short-term memory network......Page 164 LSTM implementation with tensorflow......Page 166 Language modeling......Page 169 Sequence tagging......Page 171 Machine translation......Page 173 Seq2Seq inference......Page 176 Summary......Page 178 What is multimodality learning?......Page 179 Representation......Page 180 Alignment......Page 181 Co-learning......Page 182 Image captioning......Page 183 Show and tell......Page 184 Encoder......Page 185 Testing/inference......Page 186 Beam Search......Page 187 Other types of approaches......Page 188 Datasets......Page 189 Evaluation......Page 193 BLEU......Page 194 METEOR......Page 195 Attention models......Page 196 Attention in NLP......Page 197 Attention in computer vision......Page 200 The difference between hard attention and soft attention......Page 202 Visual question answering......Page 203 Multi-source based self-driving......Page 206 Summary......Page 209 Chapter 8: Deep Reinforcement Learning......Page 210 Problem setup......Page 211 Value learning-based algorithms......Page 212 Policy search-based algorithms......Page 214 Actor-critic-based algorithms......Page 215 Deep reinforcement learning......Page 216 Deep Q-network (DQN)......Page 217 Target network......Page 218 Double-DQN......Page 219 Prioritized experience delay......Page 220 Dueling DQN......Page 221 Simple reinforcement learning example......Page 222 Reinforcement learning with Q-learning example......Page 224 Summary......Page 226 Data cleaning......Page 227 Data normalization......Page 228 Random initialization......Page 229 Xavier initialization......Page 230 Learning rate......Page 231 Multi-class classification......Page 233 Preventing overfitting......Page 234 Dropout......Page 235 When to use fine-tuning......Page 236 Tricks and techniques......Page 237 Model compression......Page 238 Summary......Page 243 Generative Adversarial Networks ......Page 244 Capsule networks ......Page 246 Genomics......Page 247 Predictive medicine......Page 251 Clinical imaging......Page 252 Lip reading......Page 253 Visual reasoning......Page 255 Code synthesis......Page 257 Summary......Page 260 Other Books You May Enjoy......Page 261 Index......Page 264