Deep Learning Illustrated: A Visual, Interactive Guide to Artificial Intelligence

Description

Deep learning is one of today’s hottest fields. This approach to machine learning is achieving breakthrough results in some of today’s highest profile applications, in organizations ranging from Google to Tesla, Facebook to Apple. Thousands of technical professionals and students want to start leveraging its power, but previous books on deep learning have often been non-intuitive, inaccessible, and dry. In Deep Learning Illustrated, three world-class instructors and practitioners present a uniquely visual, intuitive, and accessible high-level introduction to the techniques and applications of deep learning. Packed with vibrant, full-color illustrations, it abstracts away much of the complexity of building deep learning models, making the field more fun to learn, and accessible to a far wider audience.   Part I’s high-level overview explains what Deep Learning is, why it has become so ubiquitous, and how it relates to concepts and terminology such as Artificial Intelligence, Machine Learning, Artificial Neural Networks, and Reinforcement Learning. These opening chapters are replete with vivid illustrations, easy-to-grasp analogies, and character-focused narratives.   Building on this foundation, the authors then offer a practical reference and tutorial for applying a wide spectrum of proven deep learning techniques. Essential theory is covered with as little mathematics as possible, and illuminated with hands-on Python code. Theory is supported with practical “run-throughs” available in accompanying Jupyter notebooks, delivering a pragmatic understanding of all major deep learning approaches and their applications: machine vision, natural language processing, image generation, and videogaming.   To help readers accomplish more in less time, the authors feature several of today’s most widely-used and innovative deep learning libraries, including TensorFlow and its high-level API, Keras; PyTorch, and the recently-released high-level Coach, a TensorFlow API that abstracts away the complexity typically associated with building Deep Reinforcement Learning algorithms. Cover Title Page Copyright Page Contents Figures Tables Examples Foreword Preface Acknowledgments About the Authors Part I: Introducing Deep Learning 1 Biological and Machine Vision Biological Vision Machine Vision The Neocognitron LeNet-5 The Traditional Machine Learning Approach ImageNet and the ILSVRC AlexNet TensorFlow Playground Quick, Draw! Summary 2 Human and Machine Language Deep Learning for Natural Language Processing Deep Learning Networks Learn Representations Automatically Natural Language Processing A Brief History of Deep Learning for NLP Computational Representations of Language One-Hot Representations of Words Word Vectors Word-Vector Arithmetic word2viz Localist Versus Distributed Representations Elements of Natural Human Language Google Duplex Summary 3 Machine Art A Boozy All-Nighter Arithmetic on Fake Human Faces Style Transfer: Converting Photos into Monet (and Vice Versa) Make Your Own Sketches Photorealistic Creating Photorealistic Images from Text Image Processing Using Deep Learning Summary 4 Game-Playing Machines Deep Learning, AI, and Other Beasts Artificial Intelligence Machine Learning Representation Learning Artificial Neural Networks Deep Learning Machine Vision Natural Language Processing Three Categories of Machine Learning Problems Supervised Learning Unsupervised Learning Reinforcement Learning Deep Reinforcement Learning Video Games Board Games AlphaGo AlphaGo Zero AlphaZero Manipulation of Objects Popular Deep Reinforcement Learning Environments OpenAI Gym DeepMind Lab Unity ML-Agents Three Categories of AI Artificial Narrow Intelligence Artificial General Intelligence Artificial Super Intelligence Summary Part II: Essential Theory Illustrated 5 The (Code) Cart Ahead of the (Theory) Horse Prerequisites Installation A Shallow Network in Keras The MNIST Handwritten Digits A Schematic Diagram of the Network Loading the Data Reformatting the Data Designing a Neural Network Architecture Training a Deep Learning Model Summary 6 Artificial Neurons Detecting Hot Dogs Biological Neuroanatomy 101 The Perceptron The Hot Dog / Not Hot Dog Detector The Most Important Equation in This Book Modern Neurons and Activation Functions The Sigmoid Neuron The Tanh Neuron ReLU: Rectified Linear Units Choosing a Neuron Summary Key Concepts 7 Artificial Neural Networks The Input Layer Dense Layers A Hot Dog-Detecting Dense Network Forward Propagation Through the First Hidden Layer Forward Propagation Through Subsequent Layers The Softmax Layer of a Fast Food-Classifying Network Revisiting Our Shallow Network Summary Key Concepts 8 Training Deep Networks Cost Functions Quadratic Cost Saturated Neurons Cross-Entropy Cost Optimization: Learning to Minimize Cost Gradient Descent Learning Rate Batch Size and Stochastic Gradient Descent Escaping the Local Minimum Backpropagation Tuning Hidden-Layer Count and Neuron Count An Intermediate Net in Keras Summary Key Concepts 9 Improving Deep Networks Weight Initialization Xavier Glorot Distributions Unstable Gradients Vanishing Gradients Exploding Gradients Batch Normalization Model Generalization (Avoiding Overfitting) L1 and L2 Regularization Dropout Data Augmentation Fancy Optimizers Momentum Nesterov Momentum AdaGrad AdaDelta and RMSProp Adam A Deep Neural Network in Keras Regression TensorBoard Summary Key Concepts Part III: Interactive Applications of Deep Learning 10 Machine Vision Convolutional Neural Networks The Two-Dimensional Structure of Visual Imagery Computational Complexity Convolutional Layers Multiple Filters A Convolutional Example Convolutional Filter Hyperparameters Pooling Layers LeNet-5 in Keras AlexNet and VGGNet in Keras Residual Networks Vanishing Gradients: The Bête Noire of Deep CNNs Residual Connections ResNet Applications of Machine Vision Object Detection Image Segmentation Transfer Learning Capsule Networks Summary Key Concepts 11 Natural Language Process Preprocessing Natural Language Data Tokenization Converting All Characters to Lowercase Removing Stop Words and Punctuation Stemming Handling n-grams Preprocessing the Full Corpus Creating Word Embeddings with word2vec The Essential Theory Behind word2vec Evaluating Word Vectors Running word2vec Plotting Word Vectors The Area under the ROC Curve The Confusion Matrix Calculating the ROC AUC Metric Natural Language Classification with Familiar Networks Loading the IMDb Film Reviews Examining the IMDb Data Standardizing the Length of the Reviews Dense Network Convolutional Networks Networks Designed for Sequential Data Recurrent Neural Networks Long Short-Term Memory Units Bidirectional LSTMs Stacked Recurrent Models Seq2seq and Attention Transfer Learning in NLP Non-sequential Architectures: The Keras Functional API Summary Key Concepts 12 Generative Adversarial Networks Essential GAN Theory The Quick, Draw! Dataset The Discriminator Network The Generator Network The Adversarial Network GAN Training Summary Key Concepts 13 Deep Reinforcement Learning Essential Theory of Reinforcement Learning The Cart-Pole Game Markov Decision Processes The Optimal Policy Essential Theory of Deep Q-Learning Networks Value Functions Q-Value Functions Estimating an Optimal Q-Value Defining a DQN Agent Initialization Parameters Building the Agent’s Neural Network Model Remembering Gameplay Training via Memory Replay Selecting an Action to Take Saving and Loading Model Parameters Interacting with an OpenAI Gym Environment Hyperparameter Optimization with SLM Lab Agents Beyond DQN Policy Gradients and the REINFORCE Algorithm The Actor-Critic Algorithm Summary Key Concepts Part IV: You and AI 14 Moving Forward with Your Own Deep Learning Projects Ideas for Deep Learning Projects Machine Vision and GANs Natural Language Processing Deep Reinforcement Learning Converting an Existing Machine Learning Project Resources for Further Projects Socially Beneficial Projects The Modeling Process, Including Hyperparameter Tuning Automation of Hyperparameter Search Deep Learning Libraries Keras and TensorFlow PyTorch MXNet, CNTK, Caffe, and So On Software 2.0 Approaching Artificial General Intelligence Summary Part V: Appendices Appendix A: Formal Neural Network Notation Appendix B: Backpropagation Appendix C: PyTorch PyTorch Features Autograd System Define-by-Run Framework PyTorch Versus TensorFlow PyTorch in Practice PyTorch Installation The Fundamental Units Within PyTorch Building a Deep Neural Network in PyTorch Index A B C D E F G H I J K L M N O P Q R S T U V W X Y Z