Inquiry Guidelines
About the example files in this book
Translator's Note

CHAPTER 0 Introduction

0.1 Features of this book
0.1.1 Let's learn the basics of Python
0.1.2 Structure of this book
0.1.3 What you will be able to do after reading this book
0.1.4 Things to note when reading this book
0.1.5 Target audience for this book
0.1.6 The way of thinking of this book

CHAPTER 1 BERT Overview

1.1 What is deep learning?
1.1.1 Artificial Intelligence, Machine Learning, and Deep Learning
1.1.2 Structure of neural networks
1.1.3 Deep Learning
1.2 Overview of Natural Language Processing
1.2.1 What is Natural Language Processing?
1.2.2 Natural Language Processing Applications
1.2.3 Morphological analysis
1.2.4 Vectorizing words
1.2.5 RNN (Recurrent Neural Network)
1.2.6 Series transformation using Seq2Seq
1.2.7 If you want to learn more about natural language processing
1.3 Transformer Overview
1.3.1 What is a transformer?
1.3.2 Transformer Structure
1.4 BERT Overview
1.4.1 What is BERT?
1.4.2 BERT Training Overview
1.4.3 BERT Pretraining
1.4.4 BERT Performance
1.5 Summary

CHAPTER 2 DEVELOPMENT ENVIRONMENT

2.1 How to start Google Colab
2.1.1 Preparing Google Colab
2.1.2 How to use the laptop
2.1.3 How to handle downloaded files
2.2 Sessions and Instances
2.2.1 What are sessions and instances?
2.2.2 90-minute rule
2.2.3 12-hour rule
2.2.4 Session Management
2.3 CPU and GPU
2.3.1 What are CPU, GPU, and TPU?
2.3.2 How to use GPU
2.3.3 Performance Comparison
2.4 Various features of Google Colab
2.4.1 Text Cells
2.4.2 Scratch Code Cells
2.4.3 Code Snippet
2.4.4 Code Execution History
2.4.5 Integration with GitHub
2.5 Practice
2.5.1 Code Cell Manipulation
2.5.2 Manipulating Text Cells
2.5.3 Changing and deleting cell positions
2.6 Summary

CHAPTER 3: Simple Deep Learning with PyTorch

3.1 Implementation Overview
3.1.1 Learning parameters and hyperparameters
3.1.2 Forward and backpropagation
3.1.3 Implementation Order
3.2 Tensor
3.2.1 Verifying the Package
3.2.2 Creating a Tensor
3.2.3 Interconversion between NumPy arrays and tensors
3.2.4 Accessing a portion of a tensor by specifying a range
3.2.5 Tensor Operations
3.2.6 Transforming Tensors
3.2.7 Calculating various statistics
3.2.8 Exercise: Operations between Tensors
3.2.9 Example Answer
3.3 Activation function
3.3.1 Sigmoid function
3.3.2 tanh
3.3.3 ReLU
3.3.4 Identity function
3.3.5 Softmax function
3.4 Loss function
3.4.1 Mean square error
3.4.2 Cross entropy error
3.5 Optimization Algorithm
3.5.1 Gradient and Gradient Descent Algorithms
3.5.2 Optimization Algorithm Overview
3.5.3 SGD
3.5.4 Momentum
3.5.5 AdaGrad
3.5.6 RMSProp
3.5.7 Adam
3.6 Epochs and Batch
3.6.1 Epoch and Batch
3.6.2 Batch Learning
3.6.3 Online Learning
3.6.4 Mini-batch learning
3.6.5 Learning Example
3.7 Simple Deep Learning Implementation
3.7.1 Handwritten Character Image Recognition
3.7.2 Splitting data into training and testing sets
3.7.3 Building the Model
3.7.4 Learning
3.7.5 Error Trend
3.7.6 Correct answer rate
3.7.7 Predictions using the trained model
3.8 Practice
3.8.1 Splitting data into training and testing sets
3.8.2 Building the Model
3.8.3 Learning
3.8.4 Error Trend
3.8.5 Correct answer rate
3.8.6 Example Answer
3.9 Summary

CHAPTER 4: A Simple BERT Implementation

4.1 Transformers Overview
4.1.1 What are Transformers?
4.1.2 Classes that make up Transformers
4.1.3 BERT model
4.2 Transformers Basics
4.2.1 Installing the Library
4.2.2 Transformers model: Masking parts of sentences
4.2.3 Transformers Model: Sentence Classification
4.2.4 PreTrainedModel Inheritance
4.2.5 BERT Settings
4.2.6 Tokenizer
4.3 Simple BERT Implementation
4.3.1 Installing the Library
4.3.2 Missing Word Prediction: BertForMaskedLM
4.3.3 Determining whether sentences are continuous: BertForNextSentencePrediction
4.4 Practice
4.4.1 Installing the Library
4.4.2 Loading the tokenizer
4.4.3 Loading a Model
4.4.4 Functions that determine continuity
4.4.5 Continuity Determination
4.4.6 Example Answer
4.5 Summary

CHAPTER 5 BERT'S STRUCTURE

5.1 Full image of BERT
5.1.1 BERT Training
5.1.2 BERT model
5.1.3 BERT input
5.1.4 BERT Training
5.1.5 BERT Performance
5.2 Transformers and Attention
5.2.1 Transformer Model Overview
5.2.2 What is Attention?
5.2.3 Input and Memory
5.2.4 Calculating Attention Weights
5.2.5 Values ​​and Dot Products
5.2.6 Self-Attention and Source Target Attention
5.2.7 Multihead Attention
5.2.8 Positionally-Fully-Connected Forward Neural Network
5.2.9 Positional Encoding
5.2.10 Attention Visualization
5.3 BERT Structure
5.3.1 Installing the Library
5.3.2 BERT model structure
5.3.3 BERT Settings
5.4 Practice
5.4.1 Installing the Library
5.4.2 Structure of BertForMaskedLM
5.4.3 Structure of BertForNextSentencePrediction
5.5 Summary

CHAPTER 6: Utilizing Fine Tuning

6.1 Transfer Learning and Fine Tuning
6.1.1 What is transfer learning?
6.1.2 Transfer Learning and Fine Tuning
6.2 Simple Fine Tuning
6.2.1 Installing the Library
6.2.2 Importing a Model
6.2.3 Optimization Algorithm
6.2.4 Tokenizer Settings
6.2.5 Simple Fine Tuning
6.3 Sentiment Analysis Using Fine Tuning
6.3.1 Installing the Library
6.3.2 Loading the Model and Tokenizer
6.3.3 Loading a Dataset
6.3.4 Data Preprocessing
6.3.5 Evaluation functions
6.3.6 Setting TrainingArguments
6.3.7 Trainer Settings
6.3.8 Model Training
6.3.9 Model Evaluation
6.4 Practice
6.4.1 Installing the Library
6.4.2 Loading the Model and Tokenizer
6.4.3 Layer freezing
6.4.4 Loading a Dataset
6.4.5 Data Preprocessing
6.4.6 Evaluation functions
6.4.7 Setting TrainingArguments
6.4.8 Trainer Settings
6.4.9 Model Training
6.4.10 Model Evaluation
6.4.11 Example Answer
6.5 Summary

CHAPTER 7 Utilizing BERT

7.1 BERT Usage Examples
7.1.1 Search Engine
7.1.2 Translation
7.1.3 Text Classification
7.1.4 Text Summary
7.1.5 Other Usage Examples
7.2 BERT Japanese model
7.2.1 Models and datasets used
7.2.2 Installing the Library
7.2.3 Missing Word Prediction
7.2.4 Determining whether sentences are continuous
7.3 Japanese News Classification Using BERT
7.3.1 Dataset to use
7.3.2 Deploying training data to Google Drive
7.3.3 Installing the Library
7.3.4 Integration with Google Drive
7.3.5 Loading a Dataset
7.3.6 Data Storage
7.3.7 Loading the Model and Tokenizer
7.3.8 Data Preprocessing
7.3.9 Evaluation Functions
7.3.10 Setting TrainingArguments
7.3.11 Trainer Settings
7.3.12 Model Training
7.3.13 Model Evaluation
7.3.14 Saving the model
7.3.15 Importing a Model
7.3.16 Japanese News Category
7.4 BERT Korean model
7.4.1 Models and Datasets Used
7.4.2 Installing the Library
7.4.3 Missing Word Prediction
7.4.4 Determining whether sentences are continuous
7.5 Summary

Natural language processing technologies utilizing deep learning, such as BERT and GPT, continue to have a significant impact on the world today. BERT, based on Transformers, is attractive for its versatility, allowing it to be adapted to a variety of natural language processing tasks.
Deep learning technology will be useful in any field, as it allows you to handle a variety of tasks more efficiently and creatively.