Machine Learning Engineering with MLflow: Manage the end-to-end machine learning life cycle with MLflow

Description

Get up and running, and productive in no time with MLflow using the most effective machine learning engineering approach Key FeaturesExplore machine learning workflows for stating ML problems in a concise and clear manner using MLflowUse MLflow to iteratively develop a ML model and manage it Discover and work with the features available in MLflow to seamlessly take a model from the development phase to a production environmentBook Description MLflow is a platform for the machine learning life cycle that enables structured development and iteration of machine learning models and a seamless transition into scalable production environments. This book will take you through the different features of MLflow and how you can implement them in your ML project. You will begin by framing an ML problem and then transform your solution with MLflow, adding a workbench environment, training infrastructure, data management, model management, experimentation, and state-of-the-art ML deployment techniques on the cloud and premises. The book also explores techniques to scale up your workflow as well as performance monitoring techniques. As you progress, you'll discover how to create an operational dashboard to manage machine learning systems. Later, you will learn how you can use MLflow in the AutoML, anomaly detection, and deep learning context with the help of use cases. In addition to this, you will understand how to use machine learning platforms for local development as well as for cloud and managed environments. This book will also show you how to use MLflow in non-Python-based languages such as R and Java, along with covering approaches to extend MLflow with Plugins. By the end of this machine learning book, you will be able to produce and deploy reliable machine learning algorithms using MLflow in multiple environments. What you will learnDevelop your machine learning project locally with MLflow's different featuresSet up a centralized MLflow tracking server to manage multiple MLflow experimentsCreate a model life cycle with MLflow by creating custom modelsUse feature streams to log model results with MLflowDevelop the complete training pipeline infrastructure using MLflow featuresSet up an inference-based API pipeline and batch pipeline in MLflowScale large volumes of data by integrating MLflow with high-performance big data librariesWho this book is for This book is for data scientists, machine learning engineers, and data engineers who want to gain hands-on machine learning engineering experience and learn how they can manage an end-to-end machine learning life cycle with the help of MLflow. Intermediate-level knowledge of the Python programming language is expected. Table of ContentsIntroducing MLflowYour Machine Learning ProjectYour Data Science WorkbenchExperiment Management in MLflowManaging Models with MLflowIntroducing ML Systems ArchitectureData and Feature ManagementTraining Models with MLflowDeployment and Inference with MLflowScaling Up Your Machine Learning WorkflowPerformance MonitoringAdvanced Topics with MLflow Cover Title Copyright and Credits Table of Contents Section 1: Problem Framing and Introductions Chapter 1: Introducing MLflow Technical requirements What is MLflow? Getting started with MLflow Developing your first model with MLflow Exploring MLflow modules Exploring MLflow projects Exploring MLflow tracking Exploring MLflow Models Exploring MLflow Model Registry Summary Further reading Chapter 2: Your Machine Learning Project Technical requirements  Exploring the machine learning process Framing the machine learning problem Problem statement Success and failure definition Model output Output usage Heuristics Data layer definition Introducing the stock market prediction problem Stock movement predictor Problem statement Success and failure definition Model output Output usage Heuristics Data layer definition Sentiment analysis of market influencers Problem statement Success and failure definition Model output Output usage Heuristics Data layer definition Developing your machine learning baseline pipeline Summary Further reading Section 2: Model Development and Experimentation Chapter 3: Your Data Science Workbench Technical requirements  Understanding the value of a data science workbench Creating your own data science workbench Building our workbench Using the workbench for stock prediction Starting up your environment Updating with your own algorithms Summary Further reading Chapter 4: Experiment Management in MLflow Technical requirements Getting started with the experiments module Defining the experiment Exploring the dataset Adding experiments Steps for setting up a logistic-based classifier Comparing different models Tuning your model with hyperparameter optimization Summary Further reading Chapter 5: Managing Models with MLflow Technical requirements Understanding models in MLflow Exploring model flavors in MLflow Custom models Managing model signatures and schemas Introducing Model Registry Adding your best model to Model Registry Managing the model development life cycle Summary Further reading Section 3: Machine Learning in Production Chapter 6: Introducing ML Systems Architecture Technical requirements Understanding challenges with ML systems and projects Surveying state-of-the-art ML platforms Getting to know Michelangelo Getting to know Kubeflow Architecting the PsyStock ML platform Describing the features of the ML platform High-level systems architecture MLflow and other ecosystem tools Summary Further reading Chapter 7: Data and Feature Management Technical requirements Structuring your data pipeline project Acquiring stock data Checking data quality Generating a feature set and training data Running your end-to-end pipeline Using a feature store Summary Further reading Chapter 8: Training Models with MLflow Technical requirements Creating your training project with MLflow Implementing the training job Evaluating the model Deploying the model in the Model Registry Creating a Docker image for your training job Summary Further reading Chapter 9: Deployment and Inference with MLflow Technical requirements Starting up a local model registry Setting up a batch inference job Creating an API process for inference Deploying your models for batch scoring in Kubernetes Making a cloud deployment with AWS SageMaker Summary Further reading Section 4: Advanced Topics Chapter 10: Scaling Up Your Machine Learning Workflow Technical requirements Developing models with a Databricks Community Edition environment Integrating MLflow with Apache Spark Integrating MLflow with NVIDIA RAPIDS (GPU) Integrating MLflow with the Ray platform Summary Further reading Chapter 11: Performance Monitoring Technical requirements Overview of performance monitoring for machine learning models Monitoring data drift and model performance Monitoring data drift Monitoring target drift Monitoring model drift Infrastructure monitoring and alerting Summary Further reading Chapter 12: Advanced Topics with MLflow Technical requirements Exploring MLflow use cases with AutoML AutoML pyStock classification use case AutoML – anomaly detection in fraud Intergrating MLflow with other languages MLflow Java example MLflow R example Understanding MLflow plugins Summary Further reading Index