Deep Learning with PyTorch 1.x: Implement deep learning techniques and neural network architecture variants using Python, 2nd Edition

Description

Build and train neural network models with high speed and flexibility in text, vision, and advanced analytics using PyTorch 1.x Key Features Gain a thorough understanding of the PyTorch framework and learn to implement neural network architectures Understand GPU computing to perform heavy deep learning computations using Python Apply cutting-edge natural language processing (NLP) techniques to solve problems with textual dataBook Description PyTorch is gaining the attention of deep learning researchers and data science professionals due to its accessibility and efficiency, along with the fact that it's more native to the Python way of development. This book will get you up and running with this cutting-edge deep learning library, effectively guiding you through implementing deep learning concepts. In this second edition, you'll learn the fundamental aspects that power modern deep learning, and explore the new features of the PyTorch 1.x library. You'll understand how to solve real-world problems using CNNs, RNNs, and LSTMs, along with discovering state-of-the-art modern deep learning architectures, such as ResNet, DenseNet, and Inception. You'll then focus on applying neural networks to domains such as computer vision and NLP. Later chapters will demonstrate how to build, train, and scale a model with PyTorch and also cover complex neural networks such as GANs and autoencoders for producing text and images. In addition to this, you'll explore GPU computing and how it can be used to perform heavy computations. Finally, you'll learn how to work with deep learning-based architectures for transfer learning and reinforcement learning problems. By the end of this book, you'll be able to confidently and easily implement deep learning applications in PyTorch. What you will learn Build text classification and language modeling systems using neural networks Implement transfer learning using advanced CNN architectures Use deep reinforcement learning techniques to solve optimization problems in PyTorch Mix multiple models for a powerful ensemble model Build image classifiers by implementing CNN architectures using PyTorch Get up to speed with reinforcement learning, GANs, LSTMs, and RNNs with real-world examplesWho this book is for This book is for data scientists and machine learning engineers looking to work with deep learning algorithms using PyTorch 1.x. You will also find this book useful if you want to migrate to PyTorch 1.x. Working knowledge of Python programming and some understanding of machine learning will be helpful. Table of Contents Getting Started with Deep Learning Using PyTorch Building Blocks of Neural Networks Diving Deep into Neural Networks Deep Learning for Computer Vision Natural Language Processing with Sequence data Implementing Autoencoders Working with Generative Adversarial Networks Transfer Learning with Modern Network Architectures Deep Reinforcement Learning What Next? Cover Title Page Copyright and Credits About Packt Contributors Table of Contents Preface Section 1: Building Blocks of Deep Learning with PyTorch 1.x Chapter 1: Getting Started with Deep Learning Using PyTorch Exploring artificial intelligence The history of AI Machine learning in the real world  So, why DL? Applications of deep learning Automatic translation of text from images Object detection in self-driving cars Deep learning frameworks Why PyTorch? What's new in PyTorch v1.x? CPU versus GPU What is CUDA? Which GPUs should we use? What should you do if you don't have a GPU? Setting up PyTorch v1.x Installing PyTorch Summary Chapter 2: Building Blocks of Neural Networks What is a neural network?  Understanding the structure of neural networks Building a neural network in PyTorch PyTorch sequential neural network Building a PyTorch neural network using nn.Module Understanding PyTorch Tensors Understanding Tensor shapes and reshaping Tensors Understanding tensor operations Understanding Tensor types in PyTorch Importing our dataset as a PyTorch Tensor Training neural networks in PyTorch Summary  Section 2: Going Advanced with Deep Learning Chapter 3: Diving Deep into Neural Networks Diving into the building blocks of neural networks Layers – the fundamental blocks of neural networks Non-linear activations Sigmoid Tanh ReLU Leaky ReLU PyTorch non-linear activations The PyTorch way of building deep learning algorithms Model architecture for different machine learning problems Loss functions Optimizing network architecture Image classification using deep learning Loading data into PyTorch tensors Loading PyTorch tensors as batches Building the network architecture Training the model Summary Chapter 4: Deep Learning for Computer Vision Introduction to neural networks MNIST – getting data Building a CNN model from scratch Conv2d Pooling Nonlinear activation – ReLU View Linear layer Training the model Classifying dogs and cats – CNN from scratch Classifying dogs and cats using transfer learning Creating and exploring a VGG16 model Freezing the layers Fine-tuning VGG16 Training the VGG16 model Calculating pre-convoluted features Understanding what a CNN model learns Visualizing outputs from intermediate layers Visualizing the weights of the CNN layer Summary Chapter 5: Natural Language Processing with Sequence Data Working with text data Tokenization Converting text into characters Converting text into words N-gram representation Vectorization One-hot encoding Word embedding Training word embedding by building a sentiment classifier Downloading IMDb data and performing text tokenization Tokenizing with torchtext.data Tokenizing with torchtext.datasets Building vocabulary Generating batches of vectors Creating a network model with embedding Training the model Using pretrained word embeddings Downloading the embeddings Loading the embeddings in the model Freezing the embedding layer weights Recursive neural networks Understanding how RNN works with an example Solving text classification problem using LSTM Long-term dependency LSTM networks Preparing the data Creating batches Creating the network Training the model Convolutional network on sequence data Understanding one-dimensional convolution for sequence data Creating the network Training the model Language modeling Pretrained models Embeddings from language models Bidirectional Encoder Representations from Transformers Generative Pretrained Transformer 2 PyTorch implementations GPT-2 playground Summary Section 3: Understanding Modern Architectures in Deep Learning Chapter 6: Implementing Autoencoders Applications of autoencoders Bottleneck and loss functions Coded example – standard autoencoder Convolutional autoencoders Coded example – convolutional autoencoder Denoising autoencoders Variational autoencoders Training VAEs Coded example – VAE Restricted Boltzmann machines Training RBMs Theoretical example – RBM recommender system Coded example – RBM recommender system DBN architecture Fine-tuning Summary Further reading Chapter 7: Working with Generative Adversarial Networks Neural style transfer  Loading the data Creating the VGG model Content loss Style loss Extracting the losses  Creating a loss function for each layer Creating the optimizer Training the model Introducing GANs DCGAN Defining the generator network Transposed convolutions Batch normalization Generator Defining the discriminator network Defining loss and optimizer Training the discriminator Training the discriminator with real images Training the discriminator with fake images Training the generator network  Training the complete network Inspecting the generated images Summary Chapter 8: Transfer Learning with Modern Network Architectures Modern network architectures ResNet  Creating PyTorch datasets Creating loaders for training and validation Creating a ResNet model Extracting convolutional features Creating a custom PyTorch dataset class for the pre-convoluted features and loader Creating a simple linear model Training and validating the model Inception The Inception architecture Creating an Inception model Extracting convolutional features using register_forward_hook Creating a new dataset for the convoluted features Creating a fully connected model Training and validating the model Densely connected convolutional networks – DenseNet The _DenseBlock object The _DenseLayer object Creating a DenseNet model Extracting DenseNet features Creating a dataset and loaders Creating a fully connected model and training it Model ensembling Creating models Extracting the image features Creating a custom dataset, along with data loaders Creating an ensembling model Training and validating the model Encoder-decoder architecture  Encoder  Decoder Encoder-decoder with attention  Summary Chapter 9: Deep Reinforcement Learning Introduction to RL Model-based RL Model-free RL Comparing on-policy and off-policy Q-learning Value methods Value iteration Coded example – value iteration Policy methods Policy iteration Coded example – policy iteration Value iteration versus policy iteration Policy gradient algorithm Coded example – policy gradient algorithm Deep Q-networks DQN loss function Experience replay Coded example – DQN Double deep Q-learning Actor-critic methods Coded example – actor-critic model Asynchronous actor-critic algorithm Practical applications Summary Further reading Chapter 10: Whats Next? What's next? Overview of the book Reading and implementing research papers Interesting ideas to explore Object detection Image segmentation OpenNMT in PyTorch Allen NLP fast.ai – making neural nets uncool again Open neural network exchange How to keep yourself updated Summary Other Books You May Enjoy Index