Understanding the structure of models and designing accurate and reliable generative AI systems

As LLM commercialization accelerates, production-level implementation capabilities with accuracy, reliability, and scalability are becoming key technologies.
Developers now need to have a structural understanding of the entire LLM technology stack.
This book systematically guides you through the core concepts of generative AI, from system construction to deployment. It covers the operating principles of Transformer-based models, various prompt strategies, RAG design, fine-tuning techniques, and the use of frameworks like LangChain and LamarIndex. It will serve as a guide for developers seeking to organically connect LLM technologies from concept to application.

There are several versions of the GPT architecture for different purposes.
In later chapters, we'll cover other libraries more suitable for production environments, but here we'll introduce minGPT, a simplified version of OpenAI's GPT-2 model developed by Andrey Karpaty.
minGPT is a lightweight version of the GPT model that you can implement and experiment with directly in your repository.
/ minGPT is an educational tool developed to simply explain the GPT structure, condensed to about 300 lines of code, and uses the PyTorch library.
Its simple structure makes it useful for deeply understanding the inner workings of GPT family models, and the code includes clear explanations of each process, which aids learning.

--- p.45

ICL (in-context learning) is an approach in which a model learns by including examples or demonstrations in prompts.
Few-shot prompting is a subset of contextual learning that provides a small set of relevant examples or demonstrations to the model.
This strategy helps the model generalize and improve its performance on more complex tasks.
Few-shot prompting allows language models to learn from a small number of samples.
This adaptability allows the model to handle a wide range of tasks with only a small number of training samples.
/ In zero-shot prompting, the model generates output for a completely new task, whereas few-shot prompting improves performance by leveraging examples in context.
In this technique, the prompt often consists of several samples or inputs followed by corresponding answers.
Language models learn from these examples and apply them to answer similar questions.

--- p.92

The final step in setting up the RAG pipeline is to prepare prompts that encourage LLM to leverage the information it finds rather than relying on its own inherent knowledge.
At this stage, the model acts as an editor, reviewing the given information and organizing or generating answers that fit the prompt.
It's similar to how a lawyer, when not having all the answers memorized, looks up documents, books, and databases to answer questions and "digests" the information to come up with an answer.
Like lawyers, LLMs often seek to reduce errors (hallucinations) by referencing given resources.
/ For this to work, you need to adjust two arguments: system_prompt and user_prompt.
The main change to system_prompt is that it instructs the model to answer the question using specific chunks of information provided.
user_prompt signals the model to respond based only on the data provided between the 〈START_OF_CONTEXT〉 and 〈END_OF_CONTEXT〉 tags.
Here, we use the .join() method to concatenate the retrieved chunks into a single long string, and the .format() function to replace the first and second { } placeholders in the prompt variable with the combined context and the user's question, respectively.

--- pp.115-116

In LlamaIndex, after data collection, documents are transformed within a processing framework.
This process converts the document into smaller, more detailed units called Node objects.
Nodes are derived from the original document and contain main content, metadata, and contextual details.
LlamaIndex includes a NodeParser class that automatically converts document content into structured nodes.
The list of document objects was converted into node objects using SimpleNodeParser.

--- p.139

In RAG, query construction is the process of converting a user's question into a format compatible with various data sources.
For unstructured data, this allows us to convert questions into vector format, compare them with the vector representation of the source document, and identify the most relevant chunks.
It can also be applied to structured data such as databases by writing queries in a language such as SQL.
/ The core idea is to leverage the inherent structure of the data to answer user queries.
For example, the query 'movies about aliens in the year 1980' combines a semantic element like 'aliens' (better searched via a vector store) with a structural element like 'year == 1980'.
This process involves translating natural language queries into the database's specific query language, such as SQL (a structured query language for relational databases) or Cypher (a graph database).

--- pp.272-273

The final step of RLHF is to integrate the previously developed models.
In this step, a reward model is used to align the fine-tuned model more closely with human feedback.
During the training loop, user-defined prompts elicit responses from the fine-tuned OPT model, which are then evaluated via a reward model.
Evaluation scores are assigned based on similarity to responses likely to be generated by a human.

--- p.423