ADVANCED NATURAL LANGUAGE PROCESSING WITH TENSORFLOW 2: Build Real-world Effective Nlp... Applications Using Ner, Rnns, Seq2seq Models, Tran

Description

One-stop solution for NLP practitioners, ML developers and data scientists to build effective NLP systems that can perform real-world complicated tasks Key Features Implement deep learning algorithms such as BiLSTMS, CRFs, and many more using TensorFlow 2 Explore classical NLP techniques and libraries including parts-of-speech tagging and tokenization Learn practical applications of NLP covering the forefronts of the field like sentiment analysis and generating text Book Description In the last couple of years, there have been tremendous advances in natural language processing, and we are now moving from research labs into practical applications. Advanced Natural Language Processing comes with a perfect blend of both the theoretical and practical aspects of trending and complex NLP techniques. This book is focused on innovative applications in the field of NLP, language generation, and dialogue systems. It goes into the details of applying the concepts of text pre-processing using techniques such as tokenization, parts of speech tagging, and lemmatization using popular libraries such as Stanford NLP and SpaCy. Named Entity Recognition (NER), a cornerstone of task-oriented bots, is built from scratch using Conditional Random Fields and Viterbi Decoding on top of RNNs. Taking a practical and application-focused perspective, the book covers key emerging areas such as generating text for use in sentence completion and text summarization, bridging images and text by generating captions for images, and managing dialogue aspects of chatbot design. It also covers one of the most important reasons behind recent advances in NLP - applying transfer learning and fine-tuning using TensorFlow 2. Further, it covers practical techniques that can simplify the labelling of textual data which otherwise proves to be a costly affair. The book also has a working code for each tech piece so that you can adapt them to your use cases. By the end of this TensorFlow book, you will have an advanced knowledge of the tools, techniques and deep learning architecture used to solve complex NLP problems. What you will learn Grasp important pre-steps in building NLP applications like POS tagging Deal with vast amounts of unlabeled and small labelled Datasets in NLP Use transfer and weakly supervised learning using libraries like Snorkel Perform sentiment analysis using BERT Apply encoder-decoder NN architectures and beam search for summarizing text Use transformer models with attention to bring images and text together Build applications that generate captions and answer questions about images Use advanced TensorFlow techniques like learning rate annealing, custom layers, and custom loss functions to build the latest deep NLP models Who This Book Is For This is not an introductory book and assumes the reader is familiar with basics of NLP and has fundamental Python skills, as well as basic knowledge of machine learning and undergraduate-level calculus and linear algebra. The readers who can benefit the most from this book include: Intermediate ML developers who are familiar with the basics of supervised learning and deep learning techniques Professionals who already use TensorFlow/Python for purposes such as data science, ML, research, and analysis Cover Copyright Packt Page Contributors Table of Contents Preface Chapter 1: Essentials of NLP A typical text processing workflow Data collection and labeling Collecting labeled data Development environment setup Enabling GPUs on Google Colab Text normalization Modeling normalized data Tokenization Segmentation in Japanese Modeling tokenized data Stop word removal Modeling data with stop words removed Part-of-speech tagging Modeling data with POS tagging Stemming and lemmatization Vectorizing text Count-based vectorization Modeling after count-based vectorization Term Frequency-Inverse Document Frequency (TF-IDF) Modeling using TF-IDF features Word vectors Pretrained models using Word2Vec embeddings Summary Chapter 2: Understanding Sentiment in Natural Language with BiLSTMs Natural language understanding Bi-directional LSTMs – BiLSTMs RNN building blocks Long short-term memory (LSTM) networks Gated recurrent units (GRUs) Sentiment classification with LSTMs Loading the data Normalization and vectorization LSTM model with embeddings BiLSTM model Summary Chapter 3: Named Entity Recognition (NER) with BiLSTMs, CRFs, and Viterbi Decoding Named Entity Recognition The GMB data set Loading the data Normalizing and vectorizing data A BiLSTM model Conditional random fields (CRFs) NER with BiLSTM and CRFs Implementing the custom CRF layer, loss, and model A custom CRF model A custom loss function for NER using a CRF Implementing custom training Viterbi decoding The probability of the first word label Summary Chapter 4: Transfer Learning with BERT Transfer learning overview Types of transfer learning Domain adaptation Multi-task learning Sequential learning IMDb sentiment analysis with GloVe embeddings GloVe embeddings Loading IMDb training data Loading pre-trained GloVe embeddings Creating a pre-trained embedding matrix using GloVe Feature extraction model Fine-tuning model BERT-based transfer learning Encoder-decoder networks Attention model Transformer model The bidirectional encoder representations from transformers (BERT) model Tokenization and normalization with BERT Pre-built BERT classification model Custom model with BERT Summary Chapter 5: Generating Text with RNNs and GPT-2 Generating text – one character at a time Data loading and pre-processing Data normalization and tokenization Training the model Implementing learning rate decay as custom callback Generating text with greedy search Generative Pre-Training (GPT-2) model Generating text with GPT-2 Summary Chapter 6: Text Summarization with Seq2seq Attention and Transformer Networks Overview of text summarization Data loading and pre-processing Data tokenization and vectorization Seq2seq model with attention Encoder model Bahdanau attention layer Decoder model Training the model Generating summaries Greedy search Beam search Decoding penalties with beam search Evaluating summaries ROUGE metric evaluation Summarization – state of the art Summary Chapter 7: Multi-Modal Networks and Image Captioning with ResNets and Transformer Networks Multi-modal deep learning Vision and language tasks Image captioning MS-COCO dataset for image captioning Image processing with CNNs and ResNet50 CNNs Convolutions Pooling Regularization with dropout Residual connections and ResNets Image feature extraction with ResNet50 The Transformer model Positional encoding and masks Scaled dot-product and multi-head attention VisualEncoder Decoder Transformer Training the Transformer model with VisualEncoder Loading training data Instantiating the Transformer model Custom learning rate schedule Loss and metrics Checkpoints and masks Custom training Generating captions Improving performance and state-of-the-art models Summary Chapter 8: Weakly Supervised Learning for Classification with Snorkel Weak supervision Inner workings of weak supervision with labeling functions Using weakly supervised labels to improve IMDb sentiment analysis Pre-processing the IMDb dataset Learning a subword tokenizer A BiLSTM baseline model Tokenization and vectorizing data Training using a BiLSTM model Weakly supervised labeling with Snorkel Iterating on labeling functions Naïve-Bayes model for finding keywords Evaluating weakly supervised labels on the training set Generating unsupervised labels for unlabeled data Training BiLSTM on weakly supervised data from Snorkel Summary Chapter 9: Building Conversational AI Applications with Deep Learning Overview of conversational agents Task-oriented or slot-filling systems Question-answering and MRC conversational agents General conversational agents Summary Epilogue Installation and Setup Instructions for Code GitHub location Chapter 1 installation instructions Chapter 2 installation instructions Chapter 3 installation instructions Chapter 4 installation instructions Chapter 5 installation instructions Chapter 6 installation instructions Chapter 7 installation instructions Chapter 8 installation instructions Chapter 9 installation instructions Other Books You May Enjoy Index