Pro Deep Learning with Tensorflow: A Mathematical Approach to Advanced Artificial Intelligence in Python

Description

Deploy deep learning solutions in production with ease using TensorFlow. You'll also develop the mathematical understanding and intuition required to invent new deep learning architectures and solutions on your own. Pro Deep Learning with TensorFlow provides practical, hands-on expertise so you can learn deep learning from scratch and deploy meaningful deep learning solutions. This book will allow you to get up to speed quickly using TensorFlow and to optimize different deep learning architectures. All of the practical aspects of deep learning that are relevant in any industry are emphasized in this book. You will be able to use the prototypes demonstrated to build new deep learning applications. The code presented in the book is available in the form of iPython notebooks and scripts which allow you to try out examples and extend them in interesting ways. You will be equipped with the mathematical foundation and scientific knowledge to pursue research in this field and give back to the community. What You'll Learn Understand full stack deep learning using TensorFlow and gain a solid mathematical foundation for deep learning Deploy complex deep learning solutions in production using TensorFlow Carry out research on deep learning and perform experiments using TensorFlow Who This Book Is For Data scientists and machine learning professionals, software developers, graduate students, and open source enthusiasts Contents About the Author About the Technical Reviewer Acknowledgments Introduction Chapter 1: Mathematical Foundations Linear Algebra Vector Scalar Matrix Tensor Matrix Operations and Manipulations Addition of Two Matrices Subtraction of Two Matrices Product of Two Matrices Transpose of a Matrix Dot Product of Two Vectors Matrix Working on a Vector Linear Independence of Vectors Rank of a Matrix Identity Matrix or Operator Determinant of a Matrix Interpretation of Determinant Inverse of a Matrix Norm of a Vector Pseudo Inverse of a Matrix Unit Vector in the Direction of a Specific Vector Projection of a Vector in the Direction of Another Vector Eigen Vectors Characteristic Equation of a Matrix Power Iteration Method for Computing Eigen Vector Calculus Differentiation Gradient of a Function Successive Partial Derivatives Hessian Matrix of a Function Maxima and Minima of Functions Rules for Maxima and Minima for a Univariate Function Local Minima and Global Minima Positive Semi-Definite and Positive Definite Convex Set Convex Function Non-convex Function Multivariate Convex and Non-convex Functions Examples Taylor Series Probability Unions, Intersection, and Conditional Probability Chain Rule of Probability for Intersection of Event Mutually Exclusive Events Independence of Events Conditional Independence of Events Bayes Rule Probability Mass Function Probability Density Function Expectation of a Random Variable Variance of a Random Variable Skewness and Kurtosis Covariance Correlation Coefficient Some Common Probability Distribution Uniform Distribution Normal Distribution Multivariate Normal Distribution Bernoulli Distribution Binomial Distribution Poisson Distribution Likelihood Function Maximum Likelihood Estimate Hypothesis Testing and p Value Formulation of Machine-Learning Algorithm and Optimization Techniques Supervised Learning Linear Regression as a Supervised Learning Method Linear Regression Through Vector Space Approach Classification Hyperplanes and Linear Classifiers Unsupervised Learning Optimization Techniques for Machine Learning Gradient Descent Gradient Descent for a Multivariate Cost Function Contour Plot and Contour Lines Steepest Descent Stochastic Gradient Descent Newton’s Method Linear Curve Negative Curvature Positive Curvature Constrained Optimization Problem A Few Important Topics in Machine Learning Dimensionality Reduction Methods Principal Component Analysis When Will PCA Be Useful in Data Reduction? How Do You Know How Much Variance Is Retained by the Selected Principal Components? Singular Value Decomposition Regularization Regularization Viewed as a Constraint Optimization Problem Summary Chapter 2: Introduction to Deep-Learning Concepts and TensorFlow Deep Learning and Its Evolution Perceptrons and Perceptron Learning Algorithm Geometrical Interpretation of Perceptron Learning Limitations of Perceptron Learning Need for Non-linearity Hidden Layer Perceptrons’ Activation Function for Non-linearity Different Activation Functions for a Neuron/Perceptron Linear Activation Function Binary Threshold Activation Function Sigmoid Activation Function SoftMax Activation Function Rectified Linear Unit(ReLU) Activation Function Tanh Activation Function Learning Rule for Multi-Layer Perceptrons Network Backpropagation for Gradient Computation Generalizing the Backpropagation Method for Gradient Computation Deep Learning Versus Traditional Methods TensorFlow Common Deep-Learning Packages TensorFlow Installation TensorFlow Basics for Development Gradient-Descent Optimization Methods from a Deep-Learning Perspective Elliptical Contours Non-convexity of Cost Functions Saddle Points in the High-Dimensional Cost Functions Learning Rate in Mini-batch Approach to Stochastic Gradient Descent Optimizers in TensorFlow GradientDescentOptimizer Usage AdagradOptimizer Usage RMSprop Usage AdadeltaOptimizer Usage AdamOptimizer Usage MomentumOptimizer and Nesterov Algorithm Usage Epoch, Number of Batches, and Batch Size XOR Implementation Using TensorFlow TensorFlow Computation Graph for XOR network Linear Regression in TensorFlow Multi-class Classification with SoftMax Function Using Full-Batch Gradient Descent Multi-class Classification with SoftMax Function Using Stochastic Gradient Descent GPU Summary Chapter 3: Convolutional Neural Networks Convolution Operation Linear Time Invariant (LTI) / Linear Shift Invariant (LSI) Systems Convolution for Signals in One Dimension Analog and Digital Signals 2D and 3D signals 2D Convolution Two-dimensional Unit Step Function 2D Convolution of a Signal with an LSI System Unit Step Response 2D Convolution of an Image to Different LSI System Responses Common Image-Processing Filters Mean Filter Median Filter Gaussian Filter Gradient-based Filters Sobel Edge-Detection Filter Identity Transform Convolution Neural Networks Components of Convolution Neural Networks Input Layer Convolution Layer TensorFlow Usage Pooling Layer TensorFlow Usage Backpropagation Through the Convolutional Layer Backpropagation Through the Pooling Layers Weight Sharing Through Convolution and Its Advantages Translation Equivariance Translation Invariance Due to Pooling Dropout Layers and Regularization Convolutional Neural Network for Digit Recognition on the MNIST Dataset Convolutional Neural Network for Solving Real-World Problems Batch Normalization Different Architectures in Convolutional Neural Networks LeNet AlexNet VGG16 ResNet Transfer Learning Guidelines for Using Transfer Learning Transfer Learning with Google’s InceptionV3 Transfer Learning with Pre-trained VGG16 Summary Chapter 4: Natural Language Processing Using Recurrent Neural Networks Vector Space Model (VSM) Vector Representation of Words Word2Vec Continuous Bag of Words (CBOW) Continuous Bag of Words Implementation in TensorFlow Skip-Gram Model for Word Embedding Skip-gram Implementation in TensorFlow Global Co-occurrence Statistics–based Word Vectors GloVe Word Analogy with Word Vectors Introduction to Recurrent Neural Networks Language Modeling Predicting the Next Word in a Sentence Through RNN Versus Traditional Methods Backpropagation Through Time (BPTT) Vanishing and Exploding Gradient Problem in RNN Solution to Vanishing and Exploding Gradients Problem in RNNs Gradient Clipping Smart Initialization of the Memory-to-Memory Weight Connection Matrix and ReLU units Long Short-Term Memory (LSTM) LSTM in Reducing Exploding- and Vanishing -Gradient Problems MNIST Digit Identification in TensorFlow Using Recurrent Neural Networks Next-Word Prediction and Sentence Completion in TensorFlow Using Recurrent Neural Networks Gated Recurrent Unit (GRU) Bidirectional RNN Summary Chapter 5: Unsupervised Learning with Restricted Boltzmann Machines and Auto-encoders Boltzmann Distribution Bayesian Inference: Likelihood, Priors, and Posterior Probability Distribution Markov Chain Monte Carlo Methods for Sampling Metropolis Algorithm Restricted Boltzmann Machines Training a Restricted Boltzmann Machine Gibbs Sampling Block Gibbs Sampling Burn-in Period and Generating Samples in Gibbs Sampling Using Gibbs Sampling in Restricted Boltzmann Machines Contrastive Divergence A Restricted Boltzmann Implementation in TensorFlow Collaborative Filtering Using Restricted Boltzmann Machines Deep Belief Networks (DBNs) Auto-encoders Feature Learning Through Auto-encoders for Supervised Learning Kullback-Leibler (KL) Divergence Sparse Auto-encoders Sparse Auto-Encoder Implementation in TensorFlow Denoising Auto-Encoder A Denoising Auto-Encoder Implementation in TensorFlow PCA and ZCA Whitening Summary Chapter 6: Advanced Neural Networks Image Segmentation Binary Thresholding Method Based on Histogram of Pixel Intensities Otsu’s Method Watershed Algorithm for Image Segmentation Image Segmentation Using K-means Clustering Semantic Segmentation Sliding-Window Approach Fully Convolutional Network (FCN) Fully Convolutional Network with Downsampling and Upsampling Unpooling Max Unpooling Transpose Convolution U-Net Semantic Segmentation in TensorFlow with Fully Connected Neural Networks Image Classification and Localization Network Object Detection R-CNN Fast and Faster R-CNN Generative Adversarial Networks Maximin and Minimax Problem Zero-sum Game Minimax and Saddle Points GAN Cost Function and Training Vanishing Gradient for the Generator TensorFlow Implementation of a GAN Network TensorFlow Models’ Deployment in Production Summary Index