Building Data Engineering Fundamentals for Data Analysts with Python
 |
Description
Book Introduction
"Data Engineering Fundamentals for Data Analysts with Python" is a practical introductory book on data engineering for data analysts and job seekers.
Focusing on the ETL (Extract, Transform, Load) process, which is the core of data engineering, it covers various technologies step by step, including Python, MySQL, Docker, MongoDB, DuckDB, Milvus, Streamlit, and FastAPI.
The book has the following characteristics:
- Starting with setting up the development environment: Covers Ubuntu development environment based on MacOS/Windows, Git, Python, MySQL, and Docker.
- Practical training throughout the entire ETL process: Extract data from various data sources such as CSV/Excel/JSONL/Web Crawling/API, convert it using Python and NumPy/Pandas, and load it into MySQL/NoSQL.
- Multithreading and Optimization: Explains Python GIL limitations and bypass strategies, as well as parallel/concurrent processing techniques, with actual code.
- Database in-depth: Includes MySQL's main grammar, ERD design, partitioning, NoSQL comparison, and MongoDB utilization.
- Practical Project: Experience building an end-to-end ETL pipeline that connects DuckDB and Milvus for vector search, Streamlit-based image search, and FastAPI web app development.
Focusing on the ETL (Extract, Transform, Load) process, which is the core of data engineering, it covers various technologies step by step, including Python, MySQL, Docker, MongoDB, DuckDB, Milvus, Streamlit, and FastAPI.
The book has the following characteristics:
- Starting with setting up the development environment: Covers Ubuntu development environment based on MacOS/Windows, Git, Python, MySQL, and Docker.
- Practical training throughout the entire ETL process: Extract data from various data sources such as CSV/Excel/JSONL/Web Crawling/API, convert it using Python and NumPy/Pandas, and load it into MySQL/NoSQL.
- Multithreading and Optimization: Explains Python GIL limitations and bypass strategies, as well as parallel/concurrent processing techniques, with actual code.
- Database in-depth: Includes MySQL's main grammar, ERD design, partitioning, NoSQL comparison, and MongoDB utilization.
- Practical Project: Experience building an end-to-end ETL pipeline that connects DuckDB and Milvus for vector search, Streamlit-based image search, and FastAPI web app development.
index
# Table of Contents
Entering** ...................................................
10
Chapter 1.
Setting up the development environment (p.12)
- Ubuntu 24.02 LTS development environment setup (MacOS) ...........
12
- Setting up an Ubuntu 24.02 LTS development environment (Windows) .........
16
- Port Forwarding ......................
17
- Setting up Git .................................................
20
- Setting up Python development environment .............................
23
- MySQL Installation and Basic Configuration ........................
28
- Docker installation and configuration ............................
35
Chapter 2. Basic Python Grammar for ETL (p.41)
- ETL Overview .......................................
41
- Why use Python for ETL? .................
42
- Example 1: Collecting server log data ................
45
- Example 2: Data extraction through API connection ............
46
- Basic Python Grammar for Multithreading ..............
47
Chapter 3.
Python Multithreading (p.53)
- Basics of thread creation and management ........................
54
- Limitations of Python GIL and Parallel Processing ..................
61
- GIL bypass and efficient parallel/concurrent processing strategies........
62
- Single process vs multi-process ..................
65
Chapter 4.
Data Extraction (p.71)
- CSV file based data extraction ........................
71
- Excel-based data extraction ...........................
84
- JSONL based data extraction ..........................
92
- Web Crawling (requests, BeautifulSoup) ..........
105
- OpenWeatherMap API Crawling ..................
118
- MySQL data collection ..............................
129
Chapter 5.
MySQL Key Grammar (p.144)
- Exploring the core roles of MySQL ...........................
144
- Introduction to MySQL Tutorial .................................
147
- Create Sample Database ............................
148
- ERD concept and utilization method ...........................
151
- MySQL main grammar ................................
156
- Partitioning of apartment transaction price data by year...
174
- Efficient analysis using partitioned data ..........
179
Chapter 6.
Data Transformation (p.186)
- String Data Processing ...........
188
- Understanding and using regular expressions .......................
190
- NumPy vectorization principles, necessity, and implementation ..................
196
- Analysis of Pandas and NumPy vectorization benchmarks ............
202
- Handling missing values using NumPy ........................
209
- Data frame processing? Column transformation and type conversion...
231
- Data frame processing? Date and time processing..........
243
- Practical examples of handling time series data frames ............
265
Chapter 7.
Data Load (p.274)
- The need for load in ETL ....................
274
- Python meets databases ....................
275
- Collection and processing of practical log data ....................
294
- Storing MySQL log data in Parquet ...............
308
- Comparative Analysis of NoSQL Databases ....................
312
- Saving error logs using MongoDB ................
318
Chapter 8. Practical Examples of Building ETL Pipelines and Vector Search (p.346)
- Overall system overview ................................
346
- Multi-database access .....................
347
- Key core technologies and roles ..........................
348
- Data collection process (Extract) ........................
349
- Data transformation and loading process (Transform & Load) ......
361
- DuckDB and Milvus loading dualization ....................
393
- Streamlit Image Search Example ......................
400
- Building a web app using FastAPI and Streamlit ............
403
Entering** ...................................................
10
Chapter 1.
Setting up the development environment (p.12)
- Ubuntu 24.02 LTS development environment setup (MacOS) ...........
12
- Setting up an Ubuntu 24.02 LTS development environment (Windows) .........
16
- Port Forwarding ......................
17
- Setting up Git .................................................
20
- Setting up Python development environment .............................
23
- MySQL Installation and Basic Configuration ........................
28
- Docker installation and configuration ............................
35
Chapter 2. Basic Python Grammar for ETL (p.41)
- ETL Overview .......................................
41
- Why use Python for ETL? .................
42
- Example 1: Collecting server log data ................
45
- Example 2: Data extraction through API connection ............
46
- Basic Python Grammar for Multithreading ..............
47
Chapter 3.
Python Multithreading (p.53)
- Basics of thread creation and management ........................
54
- Limitations of Python GIL and Parallel Processing ..................
61
- GIL bypass and efficient parallel/concurrent processing strategies........
62
- Single process vs multi-process ..................
65
Chapter 4.
Data Extraction (p.71)
- CSV file based data extraction ........................
71
- Excel-based data extraction ...........................
84
- JSONL based data extraction ..........................
92
- Web Crawling (requests, BeautifulSoup) ..........
105
- OpenWeatherMap API Crawling ..................
118
- MySQL data collection ..............................
129
Chapter 5.
MySQL Key Grammar (p.144)
- Exploring the core roles of MySQL ...........................
144
- Introduction to MySQL Tutorial .................................
147
- Create Sample Database ............................
148
- ERD concept and utilization method ...........................
151
- MySQL main grammar ................................
156
- Partitioning of apartment transaction price data by year...
174
- Efficient analysis using partitioned data ..........
179
Chapter 6.
Data Transformation (p.186)
- String Data Processing ...........
188
- Understanding and using regular expressions .......................
190
- NumPy vectorization principles, necessity, and implementation ..................
196
- Analysis of Pandas and NumPy vectorization benchmarks ............
202
- Handling missing values using NumPy ........................
209
- Data frame processing? Column transformation and type conversion...
231
- Data frame processing? Date and time processing..........
243
- Practical examples of handling time series data frames ............
265
Chapter 7.
Data Load (p.274)
- The need for load in ETL ....................
274
- Python meets databases ....................
275
- Collection and processing of practical log data ....................
294
- Storing MySQL log data in Parquet ...............
308
- Comparative Analysis of NoSQL Databases ....................
312
- Saving error logs using MongoDB ................
318
Chapter 8. Practical Examples of Building ETL Pipelines and Vector Search (p.346)
- Overall system overview ................................
346
- Multi-database access .....................
347
- Key core technologies and roles ..........................
348
- Data collection process (Extract) ........................
349
- Data transformation and loading process (Transform & Load) ......
361
- DuckDB and Milvus loading dualization ....................
393
- Streamlit Image Search Example ......................
400
- Building a web app using FastAPI and Streamlit ............
403
GOODS SPECIFICS
- Date of issue: September 2, 2025
- Page count, weight, size: 408 pages | 148*210mm
- ISBN13: 9791112052599
You may also like
카테고리
korean
korean
![GQ KOREA Mark (Monthly): December [2025]](http://librairie.coreenne.fr/cdn/shop/files/8ef265dbbfbf186523ed75ba7319009d.jpg?v=1765340328&width=3840)
