Lesson 1: Teaching Machines Common Sense

Bayes' theorem, the foundation of probabilistic judgment
The act of selecting the most likely cause, MLE
The emergence of prior information overturns the results, MAP
Dictionary information used in our daily lives
Advance information to help AI make decisions
Conclusion

Lesson 2: Interpreting and Comparing Probability Distributions

Entropy tells us about uncertainty
Uncertainty is information
Entropy is also the value of information.
Entropy is ultimately a cost
Cost increases if the probability distribution is not known, cross entropy
Quantifying Additional Costs, KLD
The Incompetence of the Entropy Family 1
The Incompetence of the Entropy Family 2
Possible alternative, W distance
Cross entropy is used so primitively
Cross entropy can be used more effectively
Conclusion

Lesson 3 Raw Numbers as Probability Distributions

Softmax, that incongruous name
Why an exponential function?
There is no right answer in probability distributions.
Create custom probability distributions
Is it a cousin of the sigmoid function?
Conclusion

Lesson 4: The Objective Function That Determines Whether You Can Learn

Two similar yet different objective functions
Regression made easy thanks to logs
What's the real reason for taking the log of the target function?
The direction and pace of learning, gradient
Good gradient, bad gradient
log likelihood, which we are already familiar with
Conclusion

Lesson 5: How to Control Misaligned Learning Models

Noise is an unavoidable fate
Actively intervene in the model's learning process
This time, once again, Prior's great performance
Why small parameters are preferred
Different paths for L1 and L2
The Emergence of Batch Normalization and the Crisis of Weight Decay
Reevaluation of weight decay
Conclusion

Lesson 6: Find Hidden Variables, and Create Them If They Don't Exist

Data manipulation scenarios
Creation is easy when you know the distribution.
Hidden properties, latent variables
Separate mixed components, GMM
Explain with joint distribution, VAE
Explain with functions, not distributions, NF
Explaining the present through the past step by step: the diffusion model
Why Diffusion is Needed
Conclusion

Lesson 7: Don't Be Fooled by Performance Numbers

The beginning of classification model evaluation is the confusion matrix.
Why do they use specificity?
But why do we use precision?
Where should the threshold be set?
A method that takes all these circumstances into account, AUC
AUC in class imbalance situations
Yet, what AUC cannot show
Performance indicators of detection models, AP
Grading is possible even if there is no correct answer.
Score responses to generated images, IS
Scoring the feature distribution of generated images, FID
Conclusion

Lesson 8: The World Where AI Lives: Into High-Dimensional Space

Tracking one question
Strange phenomena occurring in high-dimensional space
High-dimensional Gaussian distributions look peculiar?
In higher dimensions, even the betrayal of probability occurs.
Worries you don't have to worry about in higher dimensions
Dimensional Curse or Dimensional Blessing
Conclusion

Lesson 9: AI is obsessed with itself, and therefore, it's an unattractive AI.

The performance is good, but I don't trust it.
Arrogance doesn't even help AI itself.
What made them proud
The point of becoming complacent
The process of becoming complacent
Transforming into a Humble AI
Confident Error, AI Hallucination
Conclusion