Automatic calculation of the first hill's differentiation
__Variable as a Step 1 Box
__A function that produces a two-stage variable
__3-step function chaining
__Step 4 Numerical Differentiation
__5-step backpropagation theory
__6-step manual backpropagation
__7-step backpropagation automation
__From 8-step recursion to loop
__9-step function more convenient
__10-step test

2nd Hill Natural Code
__11-step variable-length argument (forward propagation)
__12-step variable-length argument (improved version)
__13-step variable-length argument (backpropagation)
__14 steps Repeated use of the same variable
__15-step complex computational graph (theory)
__16-step complex computational graph (implementation)
__Step 17 Memory Management and Circular References
__18-step memory saving mode
__19-step variable usability improvement
__20-step operator overload (1)
__21-step operator overload (2)
__22-step operator overload (3)
__23-step package
__24-step differentiation of complex functions

Third High-Order Differential Calculus
__25-step calculation graph visualization (1)
__26-step calculation graph visualization (2)
__27-step Taylor series differentiation
__28-step function optimization
Optimization using Newton's method in 29 steps (manual calculation)
__30-step higher-order differentiation (preparation)
__31-step higher-order differentiation (theory)
__32-step higher-order differentiation (implementation)
__33-step Newton method optimization (automatic calculation)
__34-step sin function higher-order differentiation
__35-step higher-order differentiation calculation graph
__Uses other than 36-step higher-order differentiation

Building a Fourth Hill Neural Network
__Handling 37-step tensors
__38-step shape transformation function
__39-step sum function
__40-step broadcast function
__41-step matrix product
__42-step linear regression
__43-stage neural network
__A layer that collects step 44 parameters
__A layer that collects 45-step layers
__Parameter update performed by the 46-step Optimizer
__47-step softmax function and cross-entropy error
__48-step multi-class classification
__Step 49 Dataset class and preprocessing
DataLoader that extracts __50-step mini-batches
__51-step MNIST training

DeZero's Challenge at Hill 5
__52-step GPU support
__Saving and loading the 53-step model
__54-step dropout and test mode
__55-step CNN mechanism (1)
__56-step CNN mechanism (2)
__Step 57 conv2d function and pooling function
__58-step representative CNN (VGG16)
__Time series data processing using 59-step RNN
__60-step LSTM and data loader

Appendix A In-Place Operations (Supplement to Step 14)
Appendix B: Implementation of the get_item function (Supplement to Step 47)
Appendix C Running in Google Collaborate

★ A deep learning framework built from the ground up using Python.
Deep learning frameworks are full of amazing technologies and fun gadgets.
What structure does it have and what principles does it operate under? What technologies are used, and what underlying ideas are at work? The goal of Part 3 is to uncover these issues and truly understand them.
We've established a policy of "building from the ground up" so that you can also enjoy the technical fun in the process.
I hope you will attain a deeper understanding that is difficult to achieve simply by using tools created by others.


- 1st notice_ Laying the foundation for framework development.
- 2nd notice_ Makes code using the framework look more natural.
- 3rd notice_ Extend the framework to be able to obtain the second derivative.
Learning and understanding the 'backpropagation of backpropagation' structure will open your eyes to new possibilities in the framework.
- 4th Highland_ We have reorganized the framework for neural networks and used it to easily build neural networks.
- Chapter 5: Adds essential features for practical deep learning, including CPU support, model saving, and restoration. It also covers advanced models like CNNs and RNNs.
These topics covering deep learning applications can be solved with just a few lines of code if implemented as a framework.

★Recommendation
The title of this book, “Deep Learning from Scratch,” is so fitting.
It was a great help to me, who was always anxious about using deep learning superficially.
I highly recommend this course to anyone who wants to build a solid foundation in deep learning, as it will allow you to tackle the challenge with just a basic understanding of Python.
_Kim Hyo-rin, startup backend developer

This book will help you understand the structure of a deep learning framework while developing it, as well as provide inspiration for API design.
If you follow along closely, you can study several topics at once.
This is not a book for beginners.
If you start at the level of 'at least knowing Python', you will encounter yourself one step further.
_Moon Sang-hwan, Odd Concept, Tokyo Institute of Technology

"Deep Learning from Scratch 3" is packed with helpful content not only for deep learning but also for Python programming.
Once you understand the basic structure of deep learning frameworks through this book, it will be much easier to directly examine the code of frameworks like TensorFlow and PyTorch.
Even if you don't work in deep learning, I highly recommend this book to any Python engineer interested in artificial intelligence.
Jinhyung Park, co-founder of XL8 ​​and former Apple engineer

I recommend this to anyone who dreams of creating their own deep learning framework, as well as anyone who wants to improve their deep learning models.
Starting with automatic differentiation and building the framework from the ground up, I was able to learn the philosophy behind the goals and design of modern deep learning frameworks.
Actually, reading this book helped me understand PyTorch codes more deeply.
Don't dismiss this book as "I'm not planning on building a deep learning framework!"; as you read through the book, you'll find yourself more comfortable using popular frameworks.
Song Heon, Master's Program in Computer Vision Research, Kyushu University Graduate School

While beta-reading this third volume, I thought, "This book will dominate the introductory AI book market for the time being."
This book kills two birds with one stone: the structure of the mainstream dynamic computational graph (Define-by-Run) as of 2020, and the basic design of artificial intelligence.
If you've been struggling with the lack of connections between knowledge, this book will be a blessing.
_Time limit, VAIS artificial intelligence open chat community management team

This book will be of great help to high school students, college students, graduate students, and anyone working in the field who is just beginning to study deep learning.
The content is thorough and detailed, allowing those with only a cursory understanding of deep learning to fill in any gaps and gain a deeper understanding.
Be sure to read it!
Sangjun Ahn, author of "Python Deep Learning PyTorch"

In the post-COVID-19 era, experts who understand AI technology and can apply it to medical settings are essential.
Through this book, we hope to help you become more proficient in using popular frameworks and implement powerful deep learning models that can aid in medical diagnosis, treatment, and prevention.
I highly recommend this book to anyone who wants to apply deep learning to their field of expertise based on the theories learned in "Deep Learning from Scratch."
Lee Hyun-hoon, a specialist in Korean medicine, a doctoral candidate in the Department of Clinical Korean Medicine at Kyung Hee University

Implementing a library or framework from scratch can be challenging and arduous, but it's a great opportunity to learn the core elements of deep learning.
I've been using TensorFlow and PyTorch, and I've thought about creating my own deep learning framework at least once.
This book will take you on a long journey, from automatic differentiation to implementing neural networks.
This book will give you a great starting point for creating your own framework.
Ok Chan-ho, Nexon Korea Programmer, Microsoft MVP

This book is full of exciting adventures, giving you the immersive feeling of conquering each level of a well-crafted RPG game.
Along the way, readers will gain a thorough understanding of deep learning mechanisms and, as a bonus, learn advanced Python usage.
It's not a smooth road, but the thrill of holding a pretty expensive weapon in your hand after completing it will last a long time.
Jeon Du-yong, Professor, College of Liberal Arts, Sungwoon University; Ph.D. candidate, Department of Computer Engineering, Kyungpook National University