Chapter 1: Overview of Machine Learning Systems

1.1 When to Use Machine Learning
1.2 Understanding Machine Learning Systems
1.3 Summary

Chapter 2: Introduction to Machine Learning System Design

2.1 The Purpose of Business and Machine Learning
2.2 Machine Learning System Requirements
2.3 Iterative Process
2.4 Structuring Machine Learning Problems
2.5 Intelligence vs.
data
2.6 Summary

Chapter 3: Data Engineering Fundamentals

3.1 Data Sources
3.2 Data Format
3.3 Data Model
3.4 Data Storage Engine and Processing
3.5 Dataflow Mode
3.6 Batch Processing vs.
Stream processing
3.7 Summary

Chapter 4 Training Data

4.1 Sampling
4.2 Labeling
4.3 Class imbalance problem
4.4 Data Augmentation
4.5 Summary

Chapter 5 Feature Engineering

5.1 Learned features vs.
Engineered features
5.2 Feature Engineering Techniques
5.3 Data Leaks
5.4 How to design good features
5.5 Summary

Chapter 6: Model Development and Offline Evaluation

6.1 Model Development and Training
6.2 Model Offline Evaluation
6.3 Summary

Chapter 7: Model Deployment and Prediction Services

7.1 Common Sense About Machine Learning Deployment
7.2 Batch Prediction vs.
Online predictions
7.3 Model Compression
7.4 Machine Learning in the Cloud and at the Edge
7.5 Summary

Chapter 8: Data Distribution Shifts and Monitoring

8.1 Causes of Machine Learning System Failure
8.2 Data Distribution Shift
8.3 Monitoring and Observability
8.4 Summary

Chapter 9: Continuous Learning and Production Testing

9.1 Continuous Learning
9.2 Testing in Production
9.3 Summary

Chapter 10: Infrastructure and Tools for MLOps

10.1 Storage and Computing
10.2 Development Environment
10.3 Resource Management
10.4 Machine Learning Platform
10.5 Build vs.
purchase
10.6 Summary

Chapter 11: The Human Side of Machine Learning

11.1 User Experience
11.2 Team Structure
11.3 Responsible AI
11.4 Summary

The "real" machine learning story needed in the field.

When we think of machine learning, we usually think of model development and algorithms, but there's much more to consider when actually running machine learning in a production environment.
We need to consider the data, features, model development, evaluation, deployment, monitoring, and infrastructure that make up a machine learning system from a holistic perspective.
Since production machine learning is largely business-focused, business problem requirements and stakeholders are also important.

This book is based on Stanford's CS329S: Machine Learning Systems Design, a leading course in the emerging field of MLOps.
Author Chip Huyen, drawing on his experience deploying and operating machine learning at companies ranging from Netflix to startups, offers a variety of approaches to answering questions you've likely wondered about but struggled to find answers to.
Rather than focusing on specific tool usage, we focus on the concepts, pros, cons, and tradeoffs of each machine learning technique, providing links to a wealth of resources to help you find more information quickly.

Target audience

- Machine learning practitioners: machine learning engineers, data scientists, machine learning platform engineers, engineering managers, etc.
- Tool developers: If you want to identify areas where machine learning production is underserved and figure out how to build tools that fit into the ecosystem.
- Job seekers and students: If you are looking for a job related to machine learning.
- Technology and business leaders: Considering adopting machine learning solutions to improve products and business processes.

Key contents by chapter

Chapter 1: Overview of Machine Learning Systems

We'll explore various machine learning use cases and discuss when machine learning is appropriate and when it isn't.
We compare production-ready machine learning with research-ready machine learning and traditional software.

[Chapter 2: Introduction to Machine Learning System Design]

We explore an iterative process for defining system requirements based on business objectives and designing a machine learning system that satisfies them.
We discuss how to structure machine learning problems.

[Chapter 3: Data Engineering Fundamentals]

We'll explore the various data sources and formats used in machine learning projects.
Learn about data storage engines, major processing types, and different modes of passing data between processes.

[Chapter 4 Training Data]

We explore techniques for obtaining high-quality training data.
After exploring various sampling techniques, we discuss common challenges encountered when generating training data, including label multiplicity and class imbalance.

[Chapter 5 Feature Engineering]

We explore feature engineering techniques and key considerations.
We'll learn how to detect and prevent data leaks and discuss how to design good features.

[Chapter 6: Model Development and Offline Evaluation]

We'll explore useful tips for choosing the best algorithm for your task, then delve into various aspects of model development, including debugging, experiment tracking and versioning, distributed training, and AutoML.

[Chapter 7: Model Deployment and Prediction Services]

We explore common myths surrounding machine learning deployment.
After exploring online and batch prediction, we will explore various model compression techniques.
We discuss how to deploy models on edge devices and in the cloud.

[Chapter 8 Data Distribution Shift and Monitoring]

We discuss why machine learning models deployed in production fail.
We examine the issue of data distribution shift, a topic of much discussion in both research and practice.

Chapter 9: Continuous Learning and Production Testing

We explore how to update machine learning models to adapt to shifts in data distribution.
We'll explore what continuous learning is and its challenges, discuss model retraining frequency, and production testing.

[Chapter 10 Infrastructure and Tools for MLOps]

Learn how to set up the right infrastructure for your machine learning system, depending on your production scale and circumstances.
We discuss the four layers that make up infrastructure: storage, compute, resource management tools, machine learning platforms, and development environments.

Chapter 11: The Human Side of Machine Learning

We discuss how the probabilistic nature of machine learning models impacts user experience.
We explore organizational structures that enable members developing a system to collaborate effectively, and examine the impact of machine learning systems on society as a whole.