Generative AI with Python and PyTorch: Navigating the AI frontier with LLMs, Stable Diffusion, and next-gen AI applications

Description

Master GenAI techniques to create images and text using variational autoencoders (VAEs), generative adversarial networks (GANs), LSTMs, and large language models (LLMs) Key Features Implement real-world applications of LLMs and generative AI Fine-tune models with PEFT and LoRA to speed up training Expand your LLM toolbox with Retrieval Augmented Generation (RAG) techniques, LangChain, and LlamaIndex Purchase of the print or Kindle book includes a free eBook in PDF format Book Description Become an expert in Generative AI through immersive, hands-on projects that leverage today’s most powerful models for Natural Language Processing (NLP) and computer vision. Generative AI with Python and PyTorch is your end-to-end guide to creating advanced AI applications, made easy by Raghav Bali, a seasoned data scientist with multiple patents in AI, and Joseph Babcock, a PhD and machine learning expert. Through business-tested approaches, this book simplifies complex GenAI concepts, making learning both accessible and immediately applicable. From NLP to image generation, this second edition explores practical applications and the underlying theories that power these technologies. By integrating the latest advancements in LLMs, it prepares you to design and implement powerful AI systems that transform data into actionable intelligence. You’ll build your versatile LLM toolkit by gaining expertise in GPT-4, LangChain, RLHF, LoRA, RAG, and more. You’ll also explore deep learning techniques for image generation and apply styler transfer using GANs, before advancing to implement CLIP and diffusion models. Whether you’re generating dynamic content or developing complex AI-driven solutions, this book equips you with everything you need to harness the full transformative power of Python and AI. What you will learn Grasp the core concepts behind large language models and their capabilities Craft effective prompts using chain-of-thought, ReAct, and prompt query language to guide LLMs toward your desired outputs Understand how attention and transformers have changed NLP Optimize your diffusion models by combining them with VAEs Build text generation pipelines based on LSTMs and LLMs Leverage the power of open-source LLMs, such as Llama and Mistral, for diverse applications Who this book is for This book is for data scientists, machine learning engineers, and software developers seeking practical skills in building generative AI systems. A basic understanding of math and statistics and experience with Python coding is required. Cover Title Page Copyright Page Contributors Table of Contents Preface Chapter 1: Introduction to Generative AI: Drawing Data from Models Discriminative versus generative models Implementing generative models The rules of probability Discriminative and generative modeling, and Bayes’ theorem Why generative models? The promise of deep learning Generating images Data augmentation Style transfer and image transformation Fake news and chatbots Unique challenges of generative models Summary References Chapter 2: Building Blocks of Deep Neural Networks Perceptrons: A brain in a function From tissues to TLUs From TLUs to tuning perceptrons Multilayer perceptrons and backpropagation Backpropagation in practice The shortfalls of backpropagation Varieties of networks: convolution and recursive Networks for seeing: convolutional architectures Early CNNs AlexNet and other CNN innovations AlexNet architecture Networks for sequential data RNNs and LSTMs Transformers Building a better optimizer Gradient descent to ADAM Xavier initialization Summary References Chapter 3: The Rise of Methods for Text Generation Text representation Sparse representations (Bag of Words) Dense representations Word2vec GloVe FastText Contextual representations Text generation and the magic of LSTMs Language models Hands-on: Character-level language model Decoding strategies Greedy decoding Beam search Sampling Hands-on: Decoding strategies LSTM variants and convolutions for text Bidirectional LSTMs Convolutions and text Summary References Chapter 4: NLP 2.0: Using Transformers to Generate Text Attention Self-attention Transformers Overall architecture Multi-head self-attention Positional encodings NLP tasks and transformer architectures Encoder-only architectures Decoder-only architectures Encoder-decoder architectures DistilBERT in action Hands-on with DistilBERT Text generation with GPT Generative re-training: GPT GPT-2 Hands-on with GPT-2 GPT-3 Summary References Chapter 5: LLM Foundations Recap: Transformer architectures Updated training setup Instruction fine-tuning Hands-on: Instruction tuning Problem statement Dataset preparation Training setup Analyze the results Reinforcement Learning with Human Feedback (RLHF) Hands-on: RLHF using PPO Problem statement Dataset preparation PPO setup Reward model Training loop Analyze training results LLMs Summary Chapter 6: Open-Source LLMs The LLaMA models Exploring LLaMA 8B in Hugging Face Mixtral Dolly Falcon Grok-1 Summary References Chapter 7: Prompt Engineering Prompt engineering Prompt design fundamentals System instructions Prompt template Context preprocessing LLM parameters Prompting strategies Be clear and specific Use system instructions Break down complex tasks Provide examples Add contextual information Prompting techniques Task-specific prompting techniques Advanced prompting techniques Chain of Thought Tree of Thought ReAct Self-consistency Cross-domain prompting Adversarial prompting Jailbreaks Prompt injection and leakage Defence mechanisms Limitations of prompt engineering Summary References Chapter 8: LLM Toolbox The LangChain ecosystem Building a simple LLM application Creating an LLM chain Creating the LLM application Logging LLM results to LangSmith Creating complex applications with LangGraph Adding a chat interface Adding a vector store for RAG Adding a memory thread Adding a human interrupt Adding a search function Summary References Chapter 9: LLM Optimization Techniques Why optimize? Pre-training optimizations Data efficiency Architectural improvements Quantization and mixed precision Architectural efficiencies Mixture of experts Fine-tuning optimizations Parameter efficient fine-tuning Additive PEFT Reparameterization PEFT Inference time improvements Emerging trends and research areas Alternate architectures Specialized hardware and frameworks Small foundational models Summary References Chapter 10: Emerging Applications in Generative AI Advances in model development Improved text generation Improved reinforcement learning Model distillation New usages for LLMs Detecting hallucinations Multi-modal models AI agents Summary References Chapter 11: Neural Networks Using VAEs Creating separable encodings of images The variational objective The reparameterization trick Inverse autoregressive flow Importing CIFAR Creating the network in PyTorch Creating a Bernoulli MLP layer Creating a Gaussian MLP layer Combining subnetworks in a VAE Summary References Chapter 12: Image Generation with GANs Generative adversarial networks Discriminator model Generator model Training GANs Non-saturating generator cost Maximum likelihood game Vanilla GAN Improved GANs Deep convolutional GANs Conditional GANs Progressive GANs Overview Progressive growth-smooth fade-in Minibatch standard deviation Equalized learning rate Pixelwise normalization PyTorch GAN zoo implementation Challenges Training instability Mode collapse Uninformative loss and evaluation metrics Summary References Chapter 13: Style Transfer with GANs Pix2Pix-GAN: paired style transfer U-Net generator PatchGAN discriminator Loss Training Pix2Pix CycleGAN: unpaired style transfer Overall setup for CycleGAN Adversarial loss Cycle loss Identity loss Overall loss Hands-on Generator setup Discriminator setup GAN setup Training loop Summary References Chapter 14: Deepfakes with GANs Deepfakes overview Modes of operation Replacement Re-enactment Editing Other key feature sets The FACS 3DMM Key feature set Facial landmarks Facial landmark detection using OpenCV Facial landmark detection using Dlib Facial landmark detection using MTCNN High-level workflow Re-enactment using Pix2Pix Dataset preparation Pix2Pix GAN setup and training Results and limitations Challenges Ethical issues Technical challenges Generalization Occlusions Temporal issues Off-the-shelf implementations Summary References Chapter 15: Diffusion Models and AI Art A walk through image generation: Why we need diffusion models Pictures from noise: Using diffusion to model natural image variability Using variational inference to generate high-quality diffusion models Stable Diffusion: Generating images in latent space Running Stable Diffusion in the cloud Installing dependencies and running an example Key parameters for Stable Diffusion text-to-image generation Deep dive into the text-to-image pipeline The tokenizer Generating text embedding Generating the latent image using the VAE decoder The U-Net Summary References Packt Page Other Books You May Enjoy Index