[Part 1] Laying the Foundational Framework for the LLM

Chapter 1 LLM Map

1.1 Deep Learning and Language Modeling
__1.1.1 Deep learning that extracts data features on its own
__1.1.2 Embedding: How Deep Learning Models Represent Data
__1.1.3 Language Modeling: How Deep Learning Models Learn Language
1.2 How the Language Model Became ChatGPT
__1.2.1 From RNN to Transformer Architecture
__1.2.2 The relationship between model size and performance as seen in the GPT series
__1.2.3 The emergence of ChatGPT
1.3 The era of LLM applications begins
__1.3.1 LLM that Revolutionized the Way We Use Knowledge
__1.3.2 sLLM: Building Smaller, More Efficient Models
__1.3.3 Techniques for more efficient learning and inference
__1.3.4 Augmented Search Generation (RAG) Technology to Address the Hallucination Phenomenon of LLM
1.4 The Future of LLM: Expanding Perception and Action
1.5 Summary

Chapter 2: A Look at the Transformer Architecture, the Core of LLM

2.1 What is Transformer Architecture?
2.2 Converting text to embeddings
__2.2.1 Tokenization
__2.2.2 Converting to token embeddings
__2.2.3 Positional Encoding
2.3 Understanding Attention
__2.3.1 How People Read and Attention
__2.3.2 Understanding queries, keys, and values
__2.3.3 Attention in Code
__2.3.4 Multi-head attention
2.4 Normalization and feedforward layers
__2.4.1 Understanding Layer Normalization
__2.4.2 Feedforward layer
2.5 Encoder
2.6 Decoder
2.7 Architectures utilizing transformers such as BERT, GPT, and T5
BERT using __2.7.1 encoder
__2.7.2 GPT using decoder
__2.7.3 BART, T5 using both encoder and decoder
2.8 Main pre-learning mechanisms
__2.8.1 Causal Language Modeling
__2.8.2 Mask Language Modeling
2.9 Summary

Chapter 3: Hugging Face Transformer Library for Handling Transformer Models

3.1 What is a Hugging Face Transformer?
3.2 Exploring the Hugging Face Hub
__3.2.1 Model Hub
__3.2.2 Dataset Hub
__3.2.3 A space where model demos can be made public and available for use
3.3 Learning how to use the Hugging Face library
__3.3.1 Using the Model
__3.3.2 Using the Tokenizer
__3.3.3 Using the dataset
3.4 Training the Model
__3.4.1 Data Preparation
__3.4.2 Training using the Trainer API
__3.4.3 Training without using the trainer API
__3.4.4 Uploading the trained model
3.5 Model Inference
__3.5.1 Inference using pipelines
__3.5.2 Direct Inference
3.6 Summary

Chapter 4: Building a Model That Obeys

4.1 Passing the Coding Test: Pretraining and Fine-tuning the Map
__4.1.1 Learning Coding Concepts: Prerequisites for the LLM
__4.1.2 Practice Problem: Fine-tuning the Map
__4.1.3 Conditions for a good instruction dataset
4.2 Improving Code Readability with a Scoring Model
__4.2.1 Creating a Scoring Model Using a Preferred Dataset
__4.2.2 Reinforcement Learning: Towards High Code Readability Scores
__4.2.3 PPO: Avoiding Compensation Hacks
__4.2.4 RLHF: Cool, but if you can avoid it…
4.3 Is reinforcement learning really necessary?
__4.3.1 Rejection Sampling: What if we simply use the data with the highest scores?
__4.3.2 DPO: Training on Preferred Datasets Directly
__4.3.3 Models trained using DPO
4.4 Summary

[Part 2: Taming the LLM]

Chapter 5: GPU-Efficient Learning

5.1 Examining Data Uploaded to the GPU
__5.1.1 Data types of deep learning models
__5.1.2 Reducing Model Size with Quantization
__5.1.3 Disassembling GPU Memory
5.2 Efficiently Utilizing a Single GPU
__5.2.1 Gradient Accumulation
__5.2.2 Gradient Checkpointing
5.3 Distributed Learning and ZeRO
__5.3.1 Distributed Learning
__5.3.2 Reducing Redundant Storage in Data Parallelism (ZeRO)
5.4 Efficient Learning Method (PEFT): LoRA
__5.4.1 LoRA learning by reconfiguring only a portion of the model parameters
__5.4.2 Viewing LoRA Settings
__5.4.3 Using LoRA Learning with Code
5.5 Efficient Learning Method (PEFT): QLoRA
__5.5.1 4-bit quantization and second-order quantization
__5.5.2 Page Optimizer
__5.5.3 Using QLoRA Models with Code
5.6 Summary

Chapter 6: Studying sLLM

6.1 Text2SQL Dataset
__6.1.1 Representative Text2SQL datasets
__6.1.2 Korean dataset
__6.1.3 Using synthetic data
6.2 Preparing the Performance Evaluation Pipeline
__6.2.1 Text2SQL Evaluation Method
__6.2.2 Building an Evaluation Dataset
__6.2.3 SQL Generation Prompt
__6.2.4 GPT-4 Evaluation Prompt and Code Preparation
6.3 Hands-on: Performing Fine-Tuning
__6.3.1 Evaluating the Base Model
__6.3.2 Performing fine tuning
__6.3.3 Training Data Cleaning and Fine-Tuning
__6.3.4 Change the basic model
__6.3.5 Model Performance Comparison
6.4 Summary

Chapter 7: Making the Model Lighter

7.1 Understanding Language Model Inference
__7.1.1 How Language Models Generate Language
__7.1.2 KV cache to reduce redundant operations
__7.1.3 GPU Architecture and Optimal Batch Size
__7.1.4 Reducing KV cache memory
7.2 Reducing model size with quantization
__7.2.1 Bits and Bites
__7.2.2 GPTQ
__7.2.3 AWQ
7.3 Using Knowledge Distillation
7.4 Summary

Chapter 8 Serving sLLM

8.1 Efficient Deployment Strategies
__8.1.1 General layout (static layout)
__8.1.2 Dynamic Deployment
__8.1.3 Continuous Batch
8.2 Efficient transformer operations
__8.2.1 Flash Attention
__8.2.2 Flash Attention 2
__8.2.3 Relative Position Encoding
8.3 Efficient Inference Strategies
__8.3.1 Kernel Fusion
__8.3.2 Page Attention
__8.3.3 Speculative Decoding
8.4 Hands-on: LLM Serving Framework
__8.4.1 Offline Serving
__8.4.2 Online Serving
8.5 Summary

[Part 3] Practical Application Development Using LLM

Chapter 9: Developing LLM Applications

9.1 Augmented Search Generation (RAG)
__9.1.1 Data Storage
__9.1.2 Integrating search results into prompts
__9.1.3 Practice: Implementing RAG with RamaIndex
9.2 LLM Cache
__9.2.1 How LLM Cache Works
__9.2.2 Hands-on: Implementing the OpenAI API Cache
9.3 Data Verification
__9.3.1 Data Verification Method
__9.3.2 Data Validation Practice
9.4 Data Logging
__9.4.1 OpenAI API Logging
__9.4.2 RamaIndex Logging
9.5 Summary

Chapter 10: Compressing Data Meaning with Embedding Models

10.1 Understanding Text Embeddings
__10.1.1 Advantages of sentence embedding
__10.1.2 One-Hot Encoding
__10.1.3 Back of Wars
__10.1.4 TF-IDF
__10.1.5 Word to Back
10.2 Sentence embedding method
__10.2.1 Two ways to compute relationships between sentences
__10.2.2 Bi-encoder model structure
__10.2.3 Generating Text and Image Embeddings with Sentence-Transformers
__10.2.4 Comparing Open Source and Commercial Embedding Models
10.3 Hands-on: Implementing Semantic Search
__10.3.1 Implementing semantic search
__10.3.2 Using the Sentence-Transformers Model in RamaIndex
10.4 Combining Search Methods to Improve Performance
__10.4.1 Keyword Search Method: BM25
__10.4.2 Understanding Mutual Rank Combinations
10.5 Hands-on: Implementing Hybrid Search
__10.5.1 Implementing BM25
__10.5.2 Implementing Mutual Rank Combinations
__10.5.3 Implementing Hybrid Search
10.6 Summary

Chapter 11: Building Embedding Models Tailored to Your Data: Improving RAG

11.1 Two Ways to Improve Search Performance
11.2 Creating a Language Model as an Embedding Model
__11.2.1 Contrastive Learning
__11.2.2 Practice: Preparing for Learning
__11.2.3 Practice: Training an Embedding Model with Similar Sentence Data
11.3 Fine-tuning the embedding model
__11.3.1 Practice: Preparing for Learning
__11.3.2 Fine-tuning using MNR loss
11.4 Reordering rankings to improve search quality
11.5 Implementing an Improved RAG with Bi-Encoder and Cross-Encoder
__11.5.1 Searching with the default embedding model
__11.5.2 Searching with a Fine-Tuned Embedding Model
__11.5.3 Combining a Fine-Tuned Embedding Model with a Cross-Encoder
11.6 Summary

Chapter 12: Extending to Vector Databases: Implementing RAG

12.1 What is a vector database?
__12.1.1 Deep Learning and Vector Databases
__12.1.2 Understanding the Vector Database Terrain
12.2 How Vector Databases Work
__12.2.1 KNN Search and Its Limitations
__12.2.2 What is ANN search?
__12.2.3 Explorable Small World (NSW)
__12.2.4 Hierarchical Structure
12.3 Exercise: Understanding the Key Parameters of the HNSW Index
__12.3.1 Understanding the parameter m
__12.3.2 Understanding the ef_construction parameter
__12.3.3 Understanding the ef_search parameter
12.4 Hands-on: Implementing Vector Search with Pinecone
__12.4.1 How to use the Pinecone client
__12.4.2 Changing the vector database in the RamaIndex
12.5 Hands-on: Implementing Multimodal Search with Pinecone
__12.5.1 Dataset
__12.5.2 Practice Flow
__12.5.3 Generating Image Descriptions with GPT-4o
__12.5.4 Save Prompt
__12.5.5 Image Embedding Search
__12.5.6 Creating an image with DALL-E 3
12.6 Summary

Chapter 13: Running an LLM

13.1 MLOps
__13.1.1 Data Management
__13.1.2 Experimental Management
__13.1.3 Model Repository
__13.1.4 Model Monitoring
13.2 What is different about LLMOps?
__13.2.1 Choosing between a commercial and open-source model
__13.2.2 Changes in model optimization methods
__13.2.3 Difficulties in LLM Evaluation
13.3 Evaluating the LLM
__13.3.1 Quantitative indicators
__13.3.2 Evaluation using benchmark datasets
__13.3.3 How people directly evaluate
__13.3.4 Evaluation through LLM
__13.3.4 RAG Evaluation
13.4 Summary

[Part 4] Multimodality, Agents, and the Future of LLM

Chapter 14 Multimodal

LLM 14.1 What is a Multimodal LLM?
__14.1.1 Components of a Multimodal LLM
__14.1.2 Multimodal LLM Learning Course
14.2 Model for linking images and text: CLIP
__14.2.1 What is the CLIP model?
__14.2.2 CLIP model learning method
__14.2.3 Utilization of CLIP model and excellent performance
__14.2.4 Using the CLIP Model Directly
14.3 A model for generating images from text: DALL-E
__14.3.1 Diffusion Model Principle
__14.3.2 DALL-E model
14.4 LLaVA
__14.4.1 LLaVA's training data
__14.4.2 LLaVA model structure
__14.4.3 LLaVA 1.5
__14.4.4 LLaVA NeXT
14.5 Summary

Chapter 15 LLM Agent

15.1 What is an Agent?
__15.1.1 Agent Components
__15.1.2 The Agent's Brain
__15.1.3 Agent's Senses
__15.1.4 Agent Behavior
15.2 Types of Agent Systems
__15.2.1 Single Agent
__15.2.2 User-Agent Interaction
__15.2.3 Multi-Agent
15.3 Evaluating Agents
15.4 Hands-on: Implementing an Agent
__15.4.1 Basic AutoGen Usage
__15.4.2 RAG Agent
__15.4.3 Multimodal Agent
15.5 Summary

Chapter 16: New Architecture

16.1 Advantages and Disadvantages of Existing Architectures
16.2 SSM
__16.2.1 S4
16.3 Selection Mechanism
16.4 Mamba
__16.4.1 Mamba Performance
__16.4.2 Comparison with existing architecture
Mamba in 16.5 code

Appendix | Preparation for the Practice

A.1 How to Use Google Colab
A.2 Hugging Face Token
A.3 OpenAI Token

| What this book covers |

- Transformer architecture, the core of LLM
- How to create a ChatGPT: Map fine-tuning and RLHF
- Learn more about open source LLM with your own data
- Lightweight model for LLM application operation
- Implementation and improvement of RAG using Rama Index
- Multimodal LLM that processes both images and voices
- Agent architecture combining long-term memory and tools in LLM

| Target audience for this book |

- Developers who want to develop AI applications using LLM
- Developers who are curious about the principles and underlying technologies of the model rather than simply utilizing the LLM API.
- Students and job seekers who want to become AI engineers
- Graduate students who want to organize LLM-related papers and technologies in a short period of time

| Download the GitHub practice code |

The practice code can be found in the book's GitHub repository (https://github.com/onlybooks/llm).
Code from GitHub can be leveraged in Google Colab in two ways:

1.
Upload from local: Clone the code from GitHub to your local environment or download it as a compressed file, then open the notebook file (ipynb) in the practice folder you want to proceed with in Google Colab.
You can proceed with the wet.
2.
Open with GitHub URL: When you select Open Notebook in Google Colab (Ctrl+O), you can open the practice notebook via the code URL in the GitHub tab among various ways to open the notebook.

| Code execution environment for this book |

The exercises in this book are run on Google Colab.
Google Colab is a notebook execution environment provided by Google that can be run in a browser with a UI similar to Python's Jupyter Notebook.
Google Colab also provides T4 GPU (16GB) for free use.
The free version of Google Colab has a 12-hour runtime limit and may disconnect if left unused for extended periods.

[Author's Note]

As of July 2024, when this article was written, the keywords in the AI ​​and LLM market can be said to be 'multi-modal, agent, and on-device AI.'
Multimodal refers to an AI model that processes various types of data, such as text, images, voice, and video, while an agent refers to a more advanced system in which an AI model has long-term memory and utilizes various tools, such as internet searches and code execution, to solve users' problems.
On-device AI refers to AI models running directly on the user's device, rather than in the cloud or on high-performance servers.
The advantage is that users can utilize AI models without worrying about their personal information being leaked, as their information does not leave the device.
This major trend in the LLM market can be confirmed through models and projects recently released by leading AI companies.

In May 2024, OpenAI unveiled GPT-4o, a new language model that can see, hear, and speak.
A day later, Google DeepMind unveiled Project Astra, a multimodal agent that does almost the same thing.
In June 2024, Anthropic released its Claude 3.5 Sonnet model, which outperformed OpenAI's GPT-4o.
Sonnet is the mid-level model name of Anthropic.
This makes us anticipate how excellent the upcoming high-performance model, the Claude 3.5 Opus, will be.

In June 2024, Apple announced a new name for the AI ​​capabilities running on its devices: Apple Intelligence.
Since the release of ChatGPT, Apple has shown no significant movement in AI research and development, leading many to believe that Apple is losing its leadership in the AI ​​era.
However, Apple has shown confidence by using a provocative name that means AI, which can also be read as Apple Intelligence, and the market is also showing great expectations.

To understand the key keywords mentioned so far, you need to understand the latest AI models themselves, including large language models, and how to utilize them.

This book covers both the models themselves and their application, helping readers keep up with recent trends in the AI ​​market.
The first two parts of the book (Chapters 1-8) introduce the principles of LLM and the model itself, including how to learn the model and how to make inferences, to help you understand it in depth.
Part 3 of the book (Chapters 9-13) explores the components needed to develop applications using LLM and Retrieval Augmented Generation (RAG).
Finally, Part 4 (Chapters 14-16) introduces multimodality, agents, and newly researched LLM architectures, providing a glimpse into the near future of LLM.

[Editor's Note]

Before discussing LLM application development, it is necessary to consider why LLM applications are so appealing to people.
Numerous technologies have emerged and disappeared over the years, but the major technologies still in use today have the distinction of having significantly advanced development or usage paradigms.
So, to determine whether LLM applications will become mainstream and become a vital toolbox in developers' brains, it's best to first look at what changes are taking place.


Let's start with changes on the user side.
Even now, people need to keep search in mind to acquire knowledge.
When a user enters a keyword or combination of words into a search engine, a list of matching documents magically appears.
In the beginning, people meticulously compiled the list one by one like a phone book to improve quality, but soon the era of information overload arrived, and automation inevitably took place. The most representative search engine, the AltaVista series, did not have an excellent algorithm for ranking search results, so people had to manually look through numerous lists.
Then, Google, which we still rely on to this day, came along and greatly improved search quality so that you could find what you wanted just by looking at the first page.


But recently, the focus has shifted from searching to asking questions.
In today's fast-paced, busy society, even the act of opening a list one by one has become burdensome, and the quality of search results is continuously declining compared to the past due to cases of misuse under the guise of search engine optimization. Therefore, it can be seen as matching the desire to ask a question and then get the answer I want.
In the case of ChatGPT or Google Gemini, multi-modality is supported, so you can ask and receive questions by combining images and text.

Next, we need to look at the changes in machine learning and deep learning.
Before the advent of generative AI and large language models (LLMs), most machine learning and deep learning models were domain-specific.
In other words, research and development has been conducted to properly solve narrow-scope problems so that predictions and insights can be made by utilizing data accumulated in the relevant domain.
Of course, when the world of deep learning began, big data, cloud, and GPU infrastructure already existed, but there was no technological or economic breakthrough to expand them for general purposes.
For example, in the field of images, various deep learning models have emerged, starting with digit judgment using the MNIST dataset and various class judgments using the CIFAR-10 dataset, and even YOLO, which can identify the location of objects and judge classes in real time, but they are all limited to image classification.


However, with the emergence of generative AI, it is evolving into a general-purpose model capable of performing a variety of tasks.
For example, ChatGPT is quite good at answering not only general questions but also specialized questions such as medical or legal ones, and advanced models such as GPT-4V or 4o that support multi-modality can also answer questions about tables or images.
It is also considered a versatile tool because it can generate Python code at runtime and display the results of execution in an isolated environment when complex calculations or graph output are required.
Generative AI is demonstrating diverse use cases, including Q&A, translation, classification, summarization, analysis, stylistic modification, and sentiment analysis, and is evolving beyond information and decision support to the level of autonomous reasoning.


Finally, we can't leave out the changes in development itself, which developers are likely to be most interested in.
Early programming revolved around logic. Functional programming languages ​​like LISP challenged human computational abilities based on mathematical theory. Procedural methods were introduced through Fortran, Pascal, and the C programming language. Object-oriented techniques were introduced through C++ and Java.
Meanwhile, advancements have been centered around data. SQL made it easy to manipulate structured data in a tabular format in a structured way, and with the advent of the big data era, it became possible to handle not only structured data but also semi-structured and unstructured data.


Meanwhile, with the recent emergence of generative AI, another wave of change is being detected.
A new method has emerged: using a language more closely related to people, rather than a language more closely related to computers, to implement business logic! This new technique, also known as prompt engineering, orchestrates LLM inputs to achieve desired results and adjusts the output to achieve the desired outcome.
Since prompts are closer to the language used by humans, it is difficult to guarantee accuracy, but in return, flexibility and extensibility are gained, which has created an opportunity to easily implement business logic that was previously quite difficult.

As generative AI emerges, it brings with it a number of changes, and developers must prepare to adapt.
If you are developing an application with the existing web browser (or app) - WAS (web application server) - database (relational or NoSQL) structure, you need to identify the use cases of LLM according to corporate requirements and change the architecture accordingly.
A technology of particular note here is Augmented Search Generation (RAG), which has recently been attracting significant attention in the corporate world. RAG is an application that utilizes a technology called embedding various documents and data within a company to build a knowledge base as a vector database. It then extracts the document fragments most relevant to the user's question from the knowledge base and has the LLM summarize them.


While traditional search methods based on algorithms such as TF/IDF or BM25 encode words (keywords) in a sentence using sparse vectors, the semantic search used in RAG uses dense vectors to encode the abstract meaning and relationships of words, so it can be seen as a great match with LLM, which has strengths in language processing. RAG can be seen as a good example of the concept of information entropy advocated by Claude Shannon in the 1950s being realized in earnest, and it appropriately combines the two major technologies of embedding and LLM to achieve the best performance in order to process information containing various meanings, so developers will be able to gain a lot of inspiration just by looking at this technology ecosystem itself.


This book starts with the basic architecture of LLM, tames LLM to meet application requirements, makes it lightweight for operation in limited computing environments, and lays the foundation for smooth service delivery. It then explains step-by-step how to create a representative LLM application called RAG.
It doesn't end here, but covers how to overcome difficulties encountered in actual operation, as well as advanced topics such as multi-modality and agents.
In other words, it explains the development knowledge essential for the LLM era from both theoretical and practical perspectives, so it will be a welcome relief to developers seeking to adapt to the new paradigm.
I highly recommend this to all developers who are constantly working on research and development.

- Jaeho Park / Operator of the blog "Computer vs. Book" and translator of "Clean Code: Now Python" (Bookman, 2022)