Part 1 | Neural Networks and Deep Learning

Chapter 1: Embedding, Latent Space, and Representation
1.1 Embedding
1.2 Potential space
1.3 Expression
1.4 Practice Problems
1.5 Reference

Chapter 2 Self-Directed Learning
2.1 Self-Supervised Learning vs. Transfer Learning
2.2 Using Unlabeled Data
2.3 Self-predictive and contrastive self-supervised learning
2.4 Practice Problems
2.5 Note

Chapter 3 Few-Shot Learning
3.1 Datasets and Terminology
3.2 Practice Problems

Chapter 4: Lottery Ticket Hypothesis
4.1 Lottery Ticket Training Course
4.2 Practical implications and limitations
4.3 Practice Problems
4.4 Reference

Chapter 5: Reducing Overfitting with Data
5.1 Representative methods
__5.1.1 Additional Data Collection
__5.1.2 Data Augmentation
__5.1.3 Pre-training
5.2 Other methods
5.3 Practice Problems
5.4 Reference

Chapter 6: Modifying the Model to Reduce Overfitting
6.1 General Method
__6.1.1 Regulation
__6.1.2 Small model
__6.1.3 Notes on small models
__6.1.4 Ensemble Methods
6.2 Other methods
6.3 Choosing a Regulatory Technique
6.4 Practice Problems
6.5 Reference

Chapter 7 Multi-GPU Training Paradigm
7.1 Training Paradigm
__7.1.1 Model Parallelism
__7.1.2 Data Parallelism
__7.1.3 Tensor Parallelism
__7.1.4 Pipeline Parallelization
__7.1.5 Sequence Parallelism
7.2 Recommendation
7.3 Practice Problems
7.4 Reference

Chapter 8: The Success of Transformers
8.1 Attention Mechanism
8.2 Pre-training through self-supervised learning
8.3 Large number of parameters
8.4 Easy Parallelization
8.5 Practice Problems
See 8.6

Chapter 9 Generative AI Models
9.1 Generative Modeling vs. Discriminative Modeling
9.2 Types of Deep Generative Models
__9.2.1 Energy-based model
__9.2.2 Variational Autoencoder
__9.2.3 Generative Adversarial Networks
__9.2.4 Flow-based model
__9.2.5 Autoregressive model
__9.2.6 Diffusion Model
__9.2.7 Consistency Model
9.3 Recommended
9.4 Practice Problems
See 9.5

Chapter 10: Causes of Randomness
10.1 Initializing model weights
10.2 Dataset Sampling and Shuffling
10.3 Nondeterministic algorithms
10.4 Various Runtime Algorithms
10.5 Hardware and Drivers
10.6 Randomness and Generative AI
10.7 Practice Problems
See 10.8

Part 2 | Computer Vision

Chapter 11 Number of parameters
11.1 How to calculate the number of parameters
__11.1.1 Convolutional Layer
__11.1.2 Fully connected layer
11.2 Practical reasons
11.3 Practice Problems

Chapter 12 Fully Connected Layers and Convolutional Layers
12.1 When the kernel size and input size are the same
12.2 When the kernel size is 1
12.3 Recommended
12.4 Practice Problems

Chapter 13: Large-Scale Training Sets for Vision Transformers
13.1 Inductive bias in CNNs
13.2 ViT can outperform CNN
13.3 Inductive Bias of ViT
13.4 Recommended
13.5 Practice Problems
See 13.6

Part 3 | Natural Language Processing

Chapter 14 Distribution Hypothesis
14.1 Word2vec, BERT, GPT
14.2 Is the hypothesis valid?
14.3 Practice Problems
14.4 Reference

Chapter 15: Text Data Augmentation
15.1 Replacing synonyms
15.2 Deleting words
15.3 Changing word positions
15.4 Sentence Mixing
15.5 Added noise
15.6 Reverse translation
15.7 Synthetic Data
15.8 Recommended
15.9 Practice Problems
See 15.10

Chapter 16: Self-Attention
16.1 Attention Mechanism of RNNs
16.2 Self-Attention Mechanism
16.3 Practice Problems
See 16.4

Chapter 17: Encoder-Based Transformers and Decoder-Based Transformers
17.1 Original Transformer
17.2 Encoder
17.3 Decoder
17.4 Encoder-Decoder
17.5 Note on Terminology
17.6 Latest Transformer Model
17.7 Practice Problems
See 17.8

Chapter 18: Using and Fine-Tuning Pre-Trained Transformer Models
18.1 Using Transformers for Classification Tasks
18.2 In-Context Learning, Indexing, and Prompt Tuning
18.3 PEFT
18.4 RLHF
18.5 Applying a Pretrained Model
18.6 Practice Problems
See 18.7

Chapter 19: Evaluating Generative Large-Scale Language Models
19.1 Evaluation Criteria for LLM
19.2 Congestion
19.3 BLEU score
19.4 ROUGE score
19.5 BERTScore
19.6 Proxy Evaluation Indicators
19.7 Practice Problems
See 19.8

Part 4 | Product Development and Distribution

Chapter 20: Stateless and Stateful Training
20.1 Stateless Training
20.2 Stateful Training
20.3 Practice Problems

Chapter 21: Data-Driven AI
21.1 Data-Driven AI vs. Model-Driven AI
21.2 Recommendations
21.3 Practice Problems
See 21.4

Chapter 22: Speeding Up Inference
22.1 Parallelization
22.2 Vectorization
22.3 Loop Tiling
22.4 Operator Fusion
22.5 Quantization
22.6 Practice Problems
22.7 Reference

Chapter 23: Changes in Data Distribution
23.1 Covariate Changes
23.2 Label changes
23.3 Conceptual Change
23.4 Domain Changes
23.5 Types of Data Distribution Changes
23.6 Practice Problems
23.7 Reference

Part 5 | Predictive Performance and Model Evaluation

Chapter 24: Poisson Regression and Ordinal Regression
24.1 Practice Problems

Chapter 25 Confidence Intervals
25.1 Definition of confidence intervals
25.2 Method
__25.2.1 Method 1: Normal approximation interval
__25.2.2 Method 2: Bootstrapping the Training Set
__25.2.3 Method 3: Bootstrapping Test Set Predictions
__25.2.4 Method 4: Retraining the model with a different random seed
25.3 Recommendations
25.4 Practice Problems
25.5 Reference

Chapter 26: Confidence Intervals vs. Conformal Predictions
26.1 Confidence intervals and prediction intervals
26.2 Prediction Intervals and Conformal Prediction
26.3 Prediction domain, prediction interval, and prediction set
26.4 Computing Conformal Predictions
26.5 Example of Conformal Prediction
26.6 Advantages of Conformal Prediction
26.7 Recommendations
26.8 Practice Problems
See 26.9

Chapter 27 Appropriate Measurement Indicators
27.1 Conditions
27.2 Mean square error
27.3 Cross-entropy loss
27.4 Practice Problems

Chapter 28 k in k-fold cross-validation
Tradeoffs in choosing the 28.1 k value
28.2 Determining an Appropriate Value of k
28.3 Practice Problems
See 28.4

Chapter 29: Mismatch between training and test sets
29.1 Practice Problems

Chapter 30 Limited Label Data
30.1 Improving Model Performance with Limited Label Data
__30.1.1 Labeling More Data
__30.1.2 Data Bootstrapping
__30.1.3 Transfer Learning
__30.1.4 Self-Directed Learning
__30.1.5 Active Learning
__30.1.6 Few-Shot Learning
__30.1.7 Meta-learning
__30.1.8 Weakly Supervised Learning
__30.1.9 Semi-supervised learning
__30.1.10 Self-training
__30.1.11 Multi-Task Learning
__30.1.12 Multimodal Learning
__30.1.13 Inductive Bias
30.2 Recommendations
30.3 Practice Problems
See 30.4

Appendix Practice Problems Answers

Search

"Beyond the Basics" (Chris Albon, Director of Machine Learning, Wikimedia Foundation)

If you're ready to delve deeper into machine learning, deep learning, and AI beyond the basics, this book, structured in a question-and-answer format, will help you get started quickly and easily.


This book was born from questions frequently asked of author Sebastian Rasica.
It approaches advanced topics in a direct, simple, and clear way, making them easier to understand and very interesting.
Each chapter is concise and self-contained, addressing one fundamental question about AI.
Answers your questions with clear explanations, diagrams, and practical exercises.

What this book covers

ㆍ Core Questions: Concisely answer core questions about AI and break down complex concepts into easy-to-understand chunks.
ㆍ Broad range of topics: Covers a wide range of topics, from neural network architecture and model evaluation to computer vision and natural language processing.
ㆍ Practical Application: Learn techniques such as improving model performance and fine-tuning large-scale models.

It also contains the following:

Managing various factors of randomness in neural network training
Understanding the difference between encoder and decoder architectures in large-scale language models
ㆍ Reduce overfitting by changing data and models
ㆍ Construct confidence intervals for classification models and optimize models when label data is insufficient.
ㆍ Choose from a variety of multi-GPU training paradigms and generative AI model types.
Understanding Performance Metrics for Natural Language Processing
Understanding Inductive Bias in Vision Transformers

If you're looking for the perfect book to upgrade your knowledge of machine learning, "Machine Learning Q&AI" will easily expand your knowledge beyond the basics.

[Author's Preface]

In this book, the term machine learning is used in a comprehensive sense encompassing machine learning, deep learning, and AI.
Using a unique Q&A style, each chapter is structured around questions related to key concepts in machine learning, deep learning, and AI.
All questions are accompanied by explanations, various pictures and graphs, and practice problems to aid understanding.
Many chapters also include reference material for further study.


This book covers a wide range of topics.
Gaining new insights into existing architectures, such as convolutional neural networks, will allow us to leverage this technology more effectively.
We also cover advanced techniques such as large-scale language models (LLMs) and the inner workings of vision transformers.
Even experienced machine learning researchers and technologists will find something new to add to their toolbox.

[Translator's Preface]

As the author states, this book is designed to help readers advance beyond the introductory text.
While translating, I also learned many new things.
Unlike typical machine learning books, it doesn't contain formulas or code, so anyone can read it with an easy mind.
I hope this book will help you on your ML journey.

The errata for this book will be posted on the blog (https://tensorflow.blog/ml-q-and-ai).
Please be sure to check before reading the book.
Anything related to this book is welcome.
Please let me know anytime via blog or email.

[Beta Reader Review]

This book is explained in a Q&A format with readers and through easy-to-understand illustrations. Readers, from those interested in AI to those with specialized knowledge, will be able to resolve numerous questions and gain a deeper understanding.

- Shim Hyeong-gwang, Korean ID Information, Front-end & Back-end Developer

This book covers in-depth content and is suitable for those who have acquired the basics of machine learning and deep learning. It includes practice problems for self-directed learning and reference materials for additional study.
Beyond simple theoretical explanations, it also introduces practical application cases and the latest research trends, giving readers the feeling of being at the forefront of the AI ​​field.
Overall, this is a very useful book for anyone looking to advance their knowledge in the field of machine learning.
- Sanggil Park, Software Engineer, Viv Studios

It's fun and easy, just like reading a blog post.
Instead of rigidly listing complex concepts, we explain them as if unfolding an interesting story.
I especially liked that each topic was short and separate, so I could read the parts I was curious about first.
- Hyunseok Jo, Software Engineer, Lableup Co., Ltd.

It covers not only the technical knowledge required to implement machine learning and deep learning models, but also how to utilize them from a service perspective. It also provides practical explanations of how algorithms are actually applied in industrial settings, making it a useful resource for both AI researchers and practitioners.
In particular, it stands out for logically presenting the limitations that a particular technique may not always be the best choice.

- Lee Ho-min, Freelance Machine Learning Engineer

This book consists of Q&As on key concepts in machine learning, deep learning, and AI, and suggests implications for learning beyond basic concepts, moving beyond conventional formulas and code-centered learning.
I highly recommend this book to anyone who wants to explore their learning path and expand on key concepts after getting started with machine learning.
- Seungmin Lim_cslee, software engineer

It clearly organizes the essential information you need to know to properly study machine learning.
I think it's essential to understand not just how to use AI as a tool, but also how it works and what benefits it brings.

- Kang Min-jae, Department of Electrical and Electronic Engineering, Sungkyunkwan University

Other computer-related books usually focus on coding, but this book is different.
It was organized in a way that I could easily read the content I wanted to know, which was a great help in easily understanding unfamiliar concepts.

- Kim Jin-hyeok, Dongyang High School, Information Teacher

As a data analyst and data center-related business manager, I'm constantly reading books on machine learning and data modeling.
For those working in the industry, I think this is a good book to refer to for the relevant concepts and practical application methods.
Park Kyung-ho, Data Analyst, LS Electric