[Part 1] Text Mining Basics

Chapter 1: Text Mining Basics
1.1 Definition of Text Mining
1.2 Shifts in the Text Mining Paradigm
___1.2.1 Count-based document representation
___1.2.2 Sequence-based document representation
1.3 Knowledge and tools required for text mining
___1.3.1 Natural Language Processing Techniques
___1.3.2 Statistics and Linear Algebra
___1.3.3 Visualization Techniques
___1.3.4 Machine Learning
___1.3.5 Deep Learning
1.4 Main Applications of Text Mining
___1.4.1 Document Classification
___1.4.2 Document Creation
___1.4.3 Document Summary
___1.4.4 Questions and Answers
___1.4.5 Machine Translation
___1.4.6 Topic Modeling
1.5 Practice environment and software used in this book
___1.5.1 Basic Practice Environment
___1.5.2 Natural language processing related libraries
___1.5.3 Machine Learning Related Libraries
___1.5.4 Deep Learning Related Libraries

Chapter 2: Text Preprocessing
2.1 Concept of text preprocessing
___2.1.1 Why is preprocessing necessary?
___2.1.2 Preprocessing steps
___2.1.3 Practice Configuration
2.2 Tokenization
___2.2.1 Sentence Tokenization
___2.2.2 Word Tokenization
___2.2.3 Tokenization using regular expressions
___2.2.4 Removing noise and stop words
2.3 Normalization
___2.3.1 Stem Extraction
___2.3.2 Headword Extraction
2.4 Part of Speech Tagging
___2.4.1 Understanding Parts of Speech
___2.4.2 Part-of-speech tagging using NLTK
___2.4.3 Korean Morphological Analysis and Part-of-Speech Tagging
___2.4.4 References

Chapter 3: Graphs and Word Clouds
3.1 Word Frequency Graph - What are the most frequently used words?
3.2 See the content at a glance with a word cloud
3.3 Graphs and word clouds for Korean documents

[Part 2] BOW-based Text Mining

Chapter 4: Count-Based Document Representation
4.1 Concept of count-based document representation
4.2 BOW-based count vector generation
4.3 Creating a count vector with scikit-learn
4.4 Count vector conversion of Korean text
___4.4.1 Data Download
4.5 Using count vectors
4.6 Let's improve performance with TF-IDF

Chapter 5: BOW-based Document Classification
5.1 20 Newsgroup Data Preparation and Feature Extraction
___5.1.1 Checking and Separating Datasets
___5.1.2 Count-based feature extraction
5.2 Understanding Machine Learning and Document Classification Processes
5.3 Document classification using the Naive Bayes classifier
5.4 Document Classification Using Logistic Regression Analysis
___5.4.1 Preventing Overfitting Using Ridge Regression
___5.4.2 Feature Selection Using Lasso Regression
5.5 Other document classification methods using decision trees, etc.
5.6 How to improve performance
5.7 Count-based problems and solutions using N-grams
___5.7.1 Contextual information not available through statistics
___5.7.2 Understanding N-grams
___5.7.3 Document classification using N-grams
5.8 Classification of Korean documents
___5.8.1 Predicting movie titles for the next movie review
___5.8.2 Efforts to improve performance

Chapter 6: Dimensionality Reduction
6.1 The Curse of Dimensionality and the Reasons for Dimensionality Reduction
6.2 Dimensionality Reduction Using PCA
6.3 Dimensionality Reduction and Semantic Understanding Using LSA
___6.3.1 Dimensionality Reduction and Performance Using LSA
___6.3.2 Computing semantic-based document similarity using LSA
___6.3.3 Analysis of latent topics
___6.3.4 Analysis of semantic similarity between words
6.4 Visualization and Dimensionality Reduction Using tSNE

Chapter 7: Finding Topics with Topic Modeling
7.1 Understanding Topic Modeling and LDA
___7.1.1 What is Topic Modeling?
___7.1.2 Structure of the LDA model
___7.1.3 Model Evaluation and Determination of Appropriate Number of Topics
7.2 Topic Modeling with Scikit-Learn
___7.2.1 Data Preparation
___7.2.2 Running LDA Topic Modeling
___7.2.3 Choosing the optimal number of topics
7.3 Topic Modeling Using Gensim
___7.3.1 Gensim Usage and Visualization
___7.3.2 Optimal Value Selection Using Confusion and Topic Cohesion
7.4 Finding out how topics change over time with topic trends
7.5 Dynamic Topic Modeling

Chapter 8: Sentiment Analysis
8.1 Understanding Sentiment Analysis
___8.1.1 Vocabulary-based sentiment analysis
___8.1.2 Machine Learning-Based Sentiment Analysis
8.2 Sentiment Analysis of Movie Reviews Using a Sentiment Dictionary
___8.2.1 Preparing NLTK Movie Review Data
___8.2.2 Sentiment Analysis Using TextBlob
___8.2.3 Sentiment Analysis Using AFINN
___8.2.4 Sentiment Analysis Using VADER
8.3 Machine Learning-Based Sentiment Analysis through Learning
___8.3.1 Machine Learning-Based Sentiment Analysis of NLTK Movie Reviews
___8.3.2 Machine Learning-Based Sentiment Analysis of the Next Movie Reviews
8.4 References

Chapter 9: Understanding Artificial Neural Networks and Deep Learning
9.1 Understanding Artificial Neural Networks
___9.1.1 Structure and components of artificial neural networks
___9.1.2 Learning in Artificial Neural Networks
___9.1.3 Understanding the Loss Function
___9.1.4 Gradient descent
9.2 Understanding Deep Learning
___9.2.1 What is Deep Learning?
___9.2.2 Problems with Deep Neural Networks
___9.2.3 Solutions in Deep Learning
___9.2.4 Various Deep Learning Algorithms
___9.2.5 Deep Learning Development and Utilization Environment

[Part 3] Deep Learning Techniques for Text Mining

Chapter 10: RNNs - Document Classification Using Deep Learning
10.1 Why RNN?
___10.1.1 Understanding RNNs
___10.1.2 Why RNNs are Suitable for Document Classification
___10.1.3 Application of RNN to Document Classification
10.2 Understanding Word Embeddings
___10.2.1 What is a word embedding?
___10.2.2 BOW and Document Embedding
___10.2.3 Word Embeddings and Deep Learning
10.3 Document Classification Using RNNs - NLTK Movie Review Sentiment Analysis
___10.3.1 Preparing Data for Word Embedding
___10.3.2 Classification using general neural network models other than RNNs
___10.3.3 RNN-based document classification using document order information
10.4 Performance Improvement Using LSTM, Bi-LSTM, and GRU

Chapter 11: Understanding Word2Vec, ELMo, and Doc2Vec
11.1 Word2Vec - A representative word embedding technique
___11.1.1 Principles of Word2Vec Learning
___11.1.2 Using Word2Vec - Importing the Trained Model
___11.1.3 FastText - Applying N-grams to Word Embeddings
11.2 ELMo - Distinguishing word meanings based on context
___11.2.1 Problems with Word2Vec
___11.2.2 ELMo Structure
11.3 Doc2Vec - Context-Aware Document Embedding

Chapter 12: CNN - Document Classification Using Image Classification
12.1 The emergence and operation of CNN
12.2 Document Classification Using CNN
__12.2.1 Principles of document classification using CNN
__12.2.2 Classifying Movie Reviews in NLTK Using CNN

Chapter 13: Attention and Transformers
13.1 Seq2seq: A Deep Learning Technique Starting from Translation
13.2 Improving Performance Using Attention
13.3 Self-attention and Transformers
___13.3.1 Understanding Self-Attention
___13.3.2 Structure of the transformer
___13.3.3 Self-Attention Principle of Encoder
___13.3.4 How the decoder works

Chapter 14: Understanding and Simple Applications of BERT
14.1 Why are language models important?
14.2 Theoretical Understanding of Pre-Learning Language Models
14.3 BERT Structure
14.4 Pretraining and Fine-tuning Using Language Models
14.5 Direct Use of Pretrained BERT Models
14.6 Using tokenizers and models with automatic classes

Chapter 15: Fine-tuning the BERT Pretrained Model
15.1 Preprocessing for BERT Training
15.2 Fine-tuning learning using a Transformer trainer
15.3 Fine-tuning Learning with PyTorch

Chapter 16: Using BERT on Korean Documents
16.1 Fine-tuning the multilingual BERT pretraining model
16.2 PyTorch Fine-Tuning for the KoBERT Pretrained Model

Chapter 17: Current Status of Transformer Transformation Models
17.1 Various tokenizers for transformer variant models
__17.1.1 BPE (Byte-Pair Encoding) Tokenizer
__17.1.2 WordPiece Tokenizer
__17.1.3 SentencePiece Unigram Tokenizer
__17.2 GPT-based transformer deformation model
__17.2.1 GPT-2
__17.2.2 GPT-3
__17.2.3 ChatGPT
17.3 BERT-based transformer variant model
__17.3.1 RoBERTa(Robustly Optimized BERT Pretraining Approach)
__17.3.2 ALBERT (A Lite BERT)
__17.3.3 ELECTRA(Efficiently Learning an Encoder that Classifies Token Replacements Accurately)
17.4 Transformer variant model using both encoder and decoder
__17.4.1 BART (Bidirectional and Auto-Regressive Transformers)
__17.4.2 T5 (Text-to-Text Transfer Transformer)
17.5 Current Status of Domestic Transformer Modification Models

Chapter 18: Document Summary Using the Transformer Model
18.1 Understanding Document Summaries
__18.1.1 Document Summary Performance Metric: ROUGE
__18.1.2 Document Summary Dataset and Transformer Transformation Model
18.2 Document Summarization Using Pipelines
18.3 Document Summarization Using the T5 Model and Autoclasses
18.4 Fine-tuning Learning Using the T5 Model and Trainer
18.5 Summary of Korean Documents

Chapter 19: Question Answering Using the Transformer Model
19.1 Understanding the Question-Answering System
19.2 Query Answering Using Pipelines
19.3 Question Answering Using Autoclasses
19.4 Fine-tuning question-answering learning using a trainer
19.5 Korean Q&A

What this book covers

- Text preprocessing techniques such as tokenization, stemming, word extraction, stopword processing, and part-of-speech tagging

- Draw word frequency graphs and word clouds

- Convert documents into count vectors and TF-IDF vectors, and find similarity between documents.

- Perform document classification and sentiment analysis using various machine learning/deep learning techniques.

- Convert Korean documents using KoNLPy and analyze them with various machine learning algorithms.

- Dimensionality reduction of document vectors, LDA topic modeling, dynamic topic modeling, and finding and visualizing topic trends.

- Understanding word embedding techniques such as Word2Vec, ELMo, and Doc2Vec

- Understanding and utilizing BERT, practicing fine-tuning learning using PyTorch, and practicing using BERT on Korean documents

- Understanding of pre-trained language models and various transformer variants such as GPT-2, GPT-3, chatGPT, RoBERTa, ALBERT, ELECTRA, BART, and T5.

- Document summary and question-and-answer practice using transformer models such as T5, KoBART, DistilBERT, and KoELECTRA