Part 1: Getting Started with PyTorch

Chapter 1: Artificial Intelligence and Methodology

What is artificial intelligence?
__History of Artificial Intelligence
__Areas of AI Application
machine learning system
__Supervised learning
__Unsupervised learning
__Semi-supervised learning
__Reinforcement learning
Machine Learning Architecture
__Data preparation
__modeling
__Model Evaluation
__Model deployment
MLOps

Chapter 2: Installing PyTorch

What is PyTorch?
PyTorch Features
Installing PyTorch
__PyTorch CPU Installation
__PyTorch GPU Installation
__Google Colaboratory

Chapter 3: PyTorch Basics

tensor
__Create a tensor
__Tensor properties
__dimensional transformation
__Data type setting
__Device Settings
__Device conversion
__Tensor conversion of NumPy arrays
__Tensor to NumPy array conversion
Hypothesis
__Hypothesis in Machine Learning
__Statistical hypothesis testing examples
loss function
__square error
__sum of squared errors
__mean square error
__Cross entropy
Optimization
__Gradient descent
__learning rate
__Optimization problem
__Simple Linear Regression: NumPy
Simple Linear Regression: PyTorch
Datasets and Dataloaders
__dataset
__Data Loader
__Multiple linear regression
Model/Dataset Separation
__Module class
__Nonlinear regression
__Model Evaluation
__Dataset separation
Saving and Loading Models
__Save/Load Entire Model
__Save/Load Model State
__Save/Load Checkpoint
Activation function
__binary classification
__sigmoid function
__Binary cross entropy
Binary Classification: PyTorch
__Nonlinear activation function
Forward propagation and backpropagation
__Forward propagation calculation
__Error calculation
__Backpropagation calculation
__Compare update results
perceptron
__single-layer perceptron
__Multilayer Perceptron
__Perceptron Model Practice

Chapter 4: Advanced PyTorch

Overfitting and underfitting
__Solving overfitting and underfitting problems
Batch normalization
__Normalization type
__Batch Normalization Solution
Weight initialization
__Constant initialization
__Random initialization
__Javier & Glorot Initialization
__Kaiming & Heo Reset
__Orthogonal initialization
__Weight Initialization Practice
Regularization
__L1 regularization
__L2 regularization
__weight decay
__Momentum
__Elastic Net
__Dropout
__Gradient Clipping
Data Augmentation and Transformation
__text data
__image data
Pre-trained model
__backbone
__Transfer learning
__Feature extraction and fine-tuning

Part 2 Natural Language Processing

Chapter 5: Tokenization

Word and character tokenization
__word tokenization
__Character tokenization
Morpheme tokenization
__Morpheme Dictionary
__KoNLPy
__NLTK
__spaCy
Subword tokenization
__byte pair encoding
__Wordpiece

Chapter 6: Embedding

language model
__Autoregressive language model
__Statistical language model
N-gram
TF-IDF
__word frequency
__Document Frequency
__Reverse document frequency
__TF-IDF
Word2Vec
__Word vectorization
__CBoW
__Skip-gram
__Hierarchical Softmax
__negative sampling
__Model Practice: Skip-gram
__Model Practice: Gensim
fastText
__Model Practice
recurrent neural networks
__Recurrent Neural Network
__long and short-term memory
__Model Practice
convolutional neural network
__Convolution layer
__Activation Map
__Pooling
__Fully connected layer
__Model Practice

Chapter 7: Transformers

Transformer
__Input embedding and positional encoding
__Special Token
__Transformer Encoder
__Transformer Decoder
__Model Practice
GPT
__GPT-1
__GPT-2
__GPT-3
__GPT 3.5
__GPT-4
__Model Practice
BERT
__Pre-learning method
__Model Practice
BART
__Pre-learning method
__How to fine-tune
__Model Practice
ELECTRA
__Pre-learning method
__Model Practice
T5
__Model Practice

Part 3 Computer Vision

Chapter 8: Image Classification

AlexNet
__LeNet-5 and AlexNet
__Model Training
__Model Inference
VGG
__AlexNet and VGG-16
__Model structure and data visualization
__Fine-tuning and model training
ResNet
__ResNet Features
__Model Implementation
Grad-CAM
__Class Activation Map
__Grad-CAM

Chapter 9: Object Detection

Faster R-CNN
__R-CNN
__Fast R-CNN
__Faster R-CNN
__Model learning process
__Model Practice
SSD
__Multi-scale feature maps
__Basic box
__Model learning process
__Model Practice
FCN
__Upsampling
__model structure
__Model Practice
Mask R-CNN
__Features of Pyramid Network
__Sort areas of interest
__Mask Classifier
__Model Practice
YOLO
__YOLOv1
__YOLOv2
__YOLOv3
__YOLOv4 / YOLOv5
__YOLOv6 / YOLOv7
__Model Practice: YOLOv8

Chapter 10: Vision Transformer

ViT
Comparison of Convolutional and ViT Models
__ViT's inductive bias
__ViT model
__Patch Embedding
__Encoder layer
__Model Practice
Swin Transformer
__Difference between ViT and Swin transformer
__Swin Transformer Model Structure
__Model Practice
CvT
__Convolutional token embedding
Convolutional embedding for attention
__Model Practice

Part 4 Service Modeling

Chapter 11: Deploying the Model

Model lightweighting
__quantization
__Knowledge Distillation
__Tensor decomposition
__ONNX
Model serving
__Model Serving Web Framework
__Postman
Docker deployment
__What is Docker?
__Build and Deploy
Demo application
__Streamlet
__Application distribution
__PyTorch model integration

Appendix A: PyTorch Lightning
__Model Training
__Trainer Class
Appendix B: Hugging Face
__PreTrainedConfig class
__PreTrainedModel class
__PreTrainedTokenizer class
__PreTrainedFeatureExtractor class
__PreTrainedImageProcessor class
__Auto class
__Trainer Class
Appendix C: PyTorch Image Model
__Create model
__Pre-trained model
__Fine tuning
Appendix D: PyTorch Compiler
Appendix E: Out of Memory

★ What this book covers ★

◎ Understanding machine learning and deep learning and in-depth practice using PyTorch
◎ Understanding tokenization of text data and practicing text embedding
◎ Understanding the Transformer model using self-attention and practicing the Transformer-based language model
◎ Understanding image classification models and verifying and visualizing their internal operating principles
◎ Understanding object detection models such as bounding box detection, semantic segmentation, and object segmentation
◎ Understand the feature pyramid network and region of interest alignment algorithm
◎ Understanding and practicing the vision transformer model that applies the transformer structure to the field of computer vision.
◎ Online API serving using web frameworks and Docker
◎ Building a deep learning demo application through a web application
◎ Practice with various management tools and techniques, including lightweighting, PyTorch Lightning, Hugging Face, and PyTorch Compiler.