Deep Learning from Scratch 4
 |
Description
Book Introduction
The "Deep Learning from Scratch" series, this time focusing on reinforcement learning!
From core reinforcement learning theory to problem solving and deep reinforcement learning, all in one book!
The characteristic of this book, as the title suggests, is ‘building from the ground up.’
Learn by implementing reinforcement learning algorithms from scratch, without relying on external libraries whose internals are unknown.
Understand the principles through pictures, solve reinforcement learning problems with math, and then implement them with code to review what you've learned.
The code is written to be as concise as possible while clearly revealing the important ideas in reinforcement learning.
It is structured so that you can experience both the difficulty and the fun of reinforcement learning by gradually increasing the level and tackling various problems.
From core reinforcement learning theory to problem solving and deep reinforcement learning, all in one book!
The characteristic of this book, as the title suggests, is ‘building from the ground up.’
Learn by implementing reinforcement learning algorithms from scratch, without relying on external libraries whose internals are unknown.
Understand the principles through pictures, solve reinforcement learning problems with math, and then implement them with code to review what you've learned.
The code is written to be as concise as possible while clearly revealing the important ideas in reinforcement learning.
It is structured so that you can experience both the difficulty and the fun of reinforcement learning by gradually increasing the level and tackling various problems.
- You can preview some of the book's contents.
Preview
index
CHAPTER 1: THE BANDIT PROBLEM
_1.1 Machine Learning Classification and Reinforcement Learning
_1.2 Bandit Issue
_1.3 Bandit Algorithm
_1.4 Bandit Algorithm Implementation
_1.5 Abnormal Issue
_1.6 Summary
CHAPTER 2 Markov Decision Processes
_2.1 What is a Markov Decision Process (MDP)?
_2.2 Environment and agent as formulas
_2.3 MDP Goals
_2.4 MDP Example
_2.5 Summary
CHAPTER 3 Bellman Equation
_3.1 Derivation of Bellman equation
_3.2 Example of Bellman Equation
_3.3 Action-Value Function (Q Function) and Bellman Equation
_3.4 Bellman optimality equation
_3.5 Example of Bellman Optimality Equation
_3.6 Summary
CHAPTER 4 DYNAMIC PROGRAMMING
_4.1 Dynamic Programming and Policy Evaluation
_4.2 Towards a bigger problem
_4.3 Policy Repetition Method
_4.4 Implementing Policy Iteration
_4.5 Value Repetition Method
_4.6 Summary
CHAPTER 5 Monte Carlo Method
_5.1 Monte Carlo Method Basics
_5.2 Evaluating Policies Using the Monte Carlo Method
_5.3 Monte Carlo method implementation
_5.4 Controlling Policy Using Monte Carlo Methods
_5.5 Off-Policy and Importance Sampling
_5.6 Summary
CHAPTER 6 TD Law
_6.1 Evaluating policies using the TD method
_6.2 SARSA
_6.3 Off-Policy SARSA
_6.4 Q Running
_6.5 Distribution Model and Sample Model
_6.6 Summary
CHAPTER 7: Neural Networks and Q-Learning
_7.1 DeZero Basics
_7.2 Linear Regression
_7.3 Neural Networks
_7.4 Q-Learning and Neural Networks
_7.5 Summary
CHAPTER 8 DQN
_8.1 OpenAI Gym
_8.2 DQN's core technology
_8.3 DQN and Atari
_8.4 DQN Extension
_8.5 Summary
CHAPTER 9 POLICY GRADIENTS
_9.1 The simplest policy gradient method
_9.2 REINFORCE
_9.3 Baseline
_9.4 Actor-Critic
_9.5 Advantages of Policy-Based Techniques
_9.6 Summary
CHAPTER 10 One Step Further
_10.1 Classification of Deep Reinforcement Learning Algorithms
_10.2 Advanced Algorithms of the Policy Gradient Series
_10.3 Advanced algorithms of the DQN series
_10.4 Case Study
_10.5 Challenges and Potentials of Deep Reinforcement Learning
_10.6 Summary
APPENDIX A Off-Policy Monte Carlo Method
A.1 Off-policy Monte Carlo method theory
A.2 Implementation of the Off-Policy Monte Carlo Method
APPENDIX B n-step TD method
APPENDIX C Understanding Double DQN
C.1 What is overfitting in DQN?
C.2 Overfitting Solutions
APPENDIX D Policy Gradient Proof
D.1 Deriving the policy slope method
D.2 Baseline Derivation
_1.1 Machine Learning Classification and Reinforcement Learning
_1.2 Bandit Issue
_1.3 Bandit Algorithm
_1.4 Bandit Algorithm Implementation
_1.5 Abnormal Issue
_1.6 Summary
CHAPTER 2 Markov Decision Processes
_2.1 What is a Markov Decision Process (MDP)?
_2.2 Environment and agent as formulas
_2.3 MDP Goals
_2.4 MDP Example
_2.5 Summary
CHAPTER 3 Bellman Equation
_3.1 Derivation of Bellman equation
_3.2 Example of Bellman Equation
_3.3 Action-Value Function (Q Function) and Bellman Equation
_3.4 Bellman optimality equation
_3.5 Example of Bellman Optimality Equation
_3.6 Summary
CHAPTER 4 DYNAMIC PROGRAMMING
_4.1 Dynamic Programming and Policy Evaluation
_4.2 Towards a bigger problem
_4.3 Policy Repetition Method
_4.4 Implementing Policy Iteration
_4.5 Value Repetition Method
_4.6 Summary
CHAPTER 5 Monte Carlo Method
_5.1 Monte Carlo Method Basics
_5.2 Evaluating Policies Using the Monte Carlo Method
_5.3 Monte Carlo method implementation
_5.4 Controlling Policy Using Monte Carlo Methods
_5.5 Off-Policy and Importance Sampling
_5.6 Summary
CHAPTER 6 TD Law
_6.1 Evaluating policies using the TD method
_6.2 SARSA
_6.3 Off-Policy SARSA
_6.4 Q Running
_6.5 Distribution Model and Sample Model
_6.6 Summary
CHAPTER 7: Neural Networks and Q-Learning
_7.1 DeZero Basics
_7.2 Linear Regression
_7.3 Neural Networks
_7.4 Q-Learning and Neural Networks
_7.5 Summary
CHAPTER 8 DQN
_8.1 OpenAI Gym
_8.2 DQN's core technology
_8.3 DQN and Atari
_8.4 DQN Extension
_8.5 Summary
CHAPTER 9 POLICY GRADIENTS
_9.1 The simplest policy gradient method
_9.2 REINFORCE
_9.3 Baseline
_9.4 Actor-Critic
_9.5 Advantages of Policy-Based Techniques
_9.6 Summary
CHAPTER 10 One Step Further
_10.1 Classification of Deep Reinforcement Learning Algorithms
_10.2 Advanced Algorithms of the Policy Gradient Series
_10.3 Advanced algorithms of the DQN series
_10.4 Case Study
_10.5 Challenges and Potentials of Deep Reinforcement Learning
_10.6 Summary
APPENDIX A Off-Policy Monte Carlo Method
A.1 Off-policy Monte Carlo method theory
A.2 Implementation of the Off-Policy Monte Carlo Method
APPENDIX B n-step TD method
APPENDIX C Understanding Double DQN
C.1 What is overfitting in DQN?
C.2 Overfitting Solutions
APPENDIX D Policy Gradient Proof
D.1 Deriving the policy slope method
D.2 Baseline Derivation
Detailed image
Publisher's Review
The shortcut to mastering reinforcement learning is to build a solid foundation!
Deep learning is a very hot field, with new algorithms and applications being announced almost every day.
With the rapid pace of development, related technologies and services are also evolving rapidly, so things that were once popular are now disappearing.
But on the other hand, there are things that have been passed down without change.
The knowledge you learn in this book is that which does not change.
The ideas and techniques that form the basis of reinforcement learning remain unchanged.
Even modern algorithms are based on ideas that have existed for a long time.
Topics such as the fundamentals of reinforcement learning, Markov decision processes, the Bellman equation, Q-learning, and neural networks will continue to be important.
Therefore, to understand current reinforcement learning and even deep reinforcement learning, it is actually a shortcut to learn the basics of reinforcement learning step by step.
We will kindly explain each formula symbol and each line of code so that you can read it even if you have only basic knowledge of Python and mathematics.
I hope you will learn the basics of reinforcement learning through this book and experience the beauty of "what doesn't change."
May the Force be with you all…
Target audience
- Readers who want to properly learn the principles of reinforcement learning
- Developers who want to understand deep learning more deeply
- Beginners in data science who have knowledge of Python and are interested in deep learning and reinforcement learning.
Key Contents
- Solving the 'bandit problem', which sequentially finds the best candidate among several candidates (Chapter 1)
- Defining a general reinforcement learning problem as a 'Markov decision process' (Chapter 2)
- Deriving the 'Bellman equation', which is key to finding the optimal answer in the Markov decision process (Chapter 3)
- Methods for solving the Bellman equation: dynamic programming (Chapter 4), Monte Carlo method (Chapter 5), and TD method (Chapter 6).
- Learn about deep learning and apply it to reinforcement learning algorithms (Chapter 7)
- Learn how to implement and extend DQN (Chapter 8)
A Different Approach from DQN: The Policy Gradient Algorithm (Chapter 9)
- A3C/DDPG/TRPO/Rainbow Algorithm and Deep Reinforcement Learning (Chapter 10)
Deep learning is a very hot field, with new algorithms and applications being announced almost every day.
With the rapid pace of development, related technologies and services are also evolving rapidly, so things that were once popular are now disappearing.
But on the other hand, there are things that have been passed down without change.
The knowledge you learn in this book is that which does not change.
The ideas and techniques that form the basis of reinforcement learning remain unchanged.
Even modern algorithms are based on ideas that have existed for a long time.
Topics such as the fundamentals of reinforcement learning, Markov decision processes, the Bellman equation, Q-learning, and neural networks will continue to be important.
Therefore, to understand current reinforcement learning and even deep reinforcement learning, it is actually a shortcut to learn the basics of reinforcement learning step by step.
We will kindly explain each formula symbol and each line of code so that you can read it even if you have only basic knowledge of Python and mathematics.
I hope you will learn the basics of reinforcement learning through this book and experience the beauty of "what doesn't change."
May the Force be with you all…
Target audience
- Readers who want to properly learn the principles of reinforcement learning
- Developers who want to understand deep learning more deeply
- Beginners in data science who have knowledge of Python and are interested in deep learning and reinforcement learning.
Key Contents
- Solving the 'bandit problem', which sequentially finds the best candidate among several candidates (Chapter 1)
- Defining a general reinforcement learning problem as a 'Markov decision process' (Chapter 2)
- Deriving the 'Bellman equation', which is key to finding the optimal answer in the Markov decision process (Chapter 3)
- Methods for solving the Bellman equation: dynamic programming (Chapter 4), Monte Carlo method (Chapter 5), and TD method (Chapter 6).
- Learn about deep learning and apply it to reinforcement learning algorithms (Chapter 7)
- Learn how to implement and extend DQN (Chapter 8)
A Different Approach from DQN: The Policy Gradient Algorithm (Chapter 9)
- A3C/DDPG/TRPO/Rainbow Algorithm and Deep Reinforcement Learning (Chapter 10)
GOODS SPECIFICS
- Date of issue: January 26, 2024
- Page count, weight, size: 372 pages | 684g | 183*235*15mm
- ISBN13: 9791169211956
- ISBN10: 116921195X
You may also like
카테고리
korean
korean
![GQ KOREA Mark (Monthly): December [2025]](http://librairie.coreenne.fr/cdn/shop/files/8ef265dbbfbf186523ed75ba7319009d.jpg?v=1765340328&width=3840)
