Skip to product information
Signal and noise
Signal and noise
Description
Book Introduction
“One of the most important books of the past decade.”
_The New York Times

The future you encounter depends on the “quality of prediction.”
The Science and Technology of Predicting Moving Targets

The Return of the Forecasting Genius: "Suggestions for More Accurate Predictions"


Immediately after Obama's victory in the 2012 U.S. presidential election, Nate Silver, who runs the political prediction blog FiveThirtyEight.com, released his hit book, The Signal and the Noise, which became an instant bestseller.
In this book, which delves into the world of statistics and probability and shakes up the paradigm of future prediction, Nate Silver explores a variety of methods for distinguishing true signals from a world of explosive data growth and noise.


Many people make all sorts of predictions, but most of them fail, leaving enormous costs to society.
The reason is simple.
Because most of us don't properly understand probability and uncertainty.
Humans struggle to detect signals, but mistake 'confident predictions' for 'more accurate predictions'.
But sometimes overconfidence leads to failure.
If we understood uncertainty more fully than we do now, our predictions would be much more accurate.
Nate Silver, who rose to prominence as a "prediction genius" with his uncanny predictions, also had his reputation shaken by Trump's victory in the 2016 presidential election.
Silver, who openly supported Hillary, predicted a relatively high probability of Trump's victory (28.6%) compared to other media outlets and research institutes, and constantly warned of the possibility of Trump turning the tables, but was unable to decisively predict a dramatic reversal.
The preface to this revised edition reveals the process at the time, as well as the thoughts, reflections, and determination of a forecasting expert during the full-blown pandemic.


Witnessing the COVID-19 pandemic that swept the world in 2020, Silver once again asks the poignant question he posed in the first edition of The Signal and the Noise.
“Why are expert predictions so often wrong, and how can we make them more reliable?” The preface to the revised edition then lists two competencies necessary for “better probabilistic thinking”: “thinking slowly” and “being wary of jumping on the bandwagon.”
The prediction is paradoxical.
Nate Silver emphasizes once again that the more humble we are about our own predictive abilities and the more willing we are to learn from our mistakes, the more information we can transform into knowledge, and the more the data we have in our hands will be transformed into insights that can foresee the future.


To be clear, my argument in this book is not that anyone should have predicted months ago that COVID-19 would be a global pandemic disaster, or that Trump's victory was certain.
My point is that these risks are largely ignored, even though there is a very high probability that something will happen that will have very important consequences.
Why is this happening?
  • You can preview some of the book's contents.
    Preview

index
|Recommendation|Your Future Is Here _ Song Gil-young
Preface to the Revised Edition: Two Suggestions for Better Probabilistic Thinking
Slow Thinking: Essential but Difficult | Beware of Riding the Wave
|Introduction|Signal and Noise
The Light and Shadow of the Information Explosion | The 'Productivity Paradox' of the Computer Age | The Promise and Pitfalls of the Big Data Era | Why the Future Always Surprises Us | Solutions for Predicting | What We'll Cover

Ⅰ.
Fundamental Questions About Forecasting
1.
Financial Crisis│The Great Failure of Predictions Surrounding the Economic Collapse
The Worst Predictions About Misfortune | “They Didn’t Want the Music to Stop” | How Credit Rating Agencies Messed Up | Act 1: The Housing Bubble, the Illusion Surrounding My Home | Act 2: Leverage, Creating House Poor | Intermission: ‘Fear’ is the New Name for ‘Greed’ | Act 3: Destruction, a New Situation Unfolds | What Four Failed Predictions Have in Common | The Formula for Failed Predictions: The Problem of Out-of-Sample Predictions | What Can We Learn?

2.
Politics│My Secret to Predicting Election Results
Are political scientists better than TV expert panels? | The right attitude for better forecasts: Be a fox | Why hedgehogs are more welcome on TV panels? | Why bad forecasts are rampant | Jumping into political forecasting | Fox's rule 1: Think probabilistically | Fox's rule 2: Make a new forecast every day | Fox's rule 3: Seek consensus | Don't trust 'magic bullet' forecasts | How to weight qualitative information | It's not easy to be objective

3.
Baseball│Why Baseball Games Are the Model for All Predictions
Building a Baseball Prediction System | The World's Richest Dataset | The Aging Curve: How Long Can a Player Play? | The Battle of 'Calculator' and 'Intuition' | Pekota vs. Scout: Scout Wins | The Invisible Factor | Running Fast and Throwing Hard Doesn't Make You Win | The Name of the Game is 'Information' | Why Pedroia's Future Was Dark | And How He Overcame Adversity | The True Lessons of Moneyball and the Future of Baseball

Ⅱ.
Hit the moving target!
4.
Weather│A Success Story in Forecasting, Advances in Weather Forecasting
Are supercomputers really useful? | A very brief history of weather forecasting | The Matrix, the birth of a new weather forecasting method | What tornadoes and basketball players have in common | The human eye still matters | The success of the weather service and the challenges of private companies | How are better forecasts made? | When competition makes forecasts worse | When error correction is needed | The eye of the storm and the cone of chaos

5.
Earthquake│Desperately, searching for a signal
What we do when the ground beneath our feet shakes | The search for the magic toad and the Holy Grail | How earthquakes work: Power-law distribution | The signal that tempts us amidst the noise | The march of failed predictions | A dilemma | Pay attention to overfitting | The Great East Japan Earthquake: A prime example of overfitting | How much can we really know about earthquakes? | The beauty of signal and noise | Science is always under test

6.
Economic Forecasting│Coping with Uncertainty, Volatility, and Bias
Why We Need to Talk About Uncertainty | Are Economists Rational? | "No One Has a Clue" | Correlation Without Causation | Economic Forecasting Is a Moving Target | Noisy Economic Data | The Butterfly Effect | When Bias Is a Rational Choice | Two Alternatives for Dealing with Bias

7.
Epidemics│All models are wrong, but some are useful.
A sequel to the 'new flu fiasco'? | When extrapolation becomes dangerous | Self-fulfilling and self-negating predictions | 'Sophisticated' simplicity | SimFlu | What to do when predictions are bound to be wrong

Ⅲ.
How to capture the future by improving the quality of your forecasts
8.
Bayes' Theorem│How to Make Less Mistakes Little by Little
How Winning Gamblers Bet | Thomas Bayes's Quirky Legacy | Probability and Progress | The Simple Math of Bayes' Theorem | The False Positive Problem | Frequentism Refutes Bayesianism | Data Without Context is Useless | Bob, the Bayesian Gambler | Bayesianism Converging on the Truth

9.
Chess│Can Computers See the Future Like Humans?
Computers that play chess | Chess, prediction, and heuristics | Kasparov and Deep Blue | The game begins | The chess player's dilemma: breadth or depth | Strategy versus tactics | The first signs of the end | Machines beat humans | How to make chess masters stumble | Bug or secret weapon? | What computers are good at | Google search engine: When large-scale trial and error is possible | Beyond the blind spots of technology

10.
Poker│How to Spot Your Opponent's Bluff
The Beginning of the Poker Dream | The Art of Hand Reading | Think Probabilistically | Schrödinger's Poker Hands | How to Confuse Your Opponent | The Pareto Principle of Prediction | The Master and the Fool: The Economics of the Poker Bubble | The Burst of the Bubble | Luck or Skill | The Swamp of Illusion | Why We Lose Our Composure | Focus on the Process, Not the Results

Ⅳ.
An invisible hand moves the world.
11.
Stocks│If an individual can never beat the market
Welcome to Bayesianland | The 'Invisible Hand' of Bayesianism | Prediction Markets: Buying and Selling Hillary Stocks | The Benefits and Limitations of Herd Forecasting | The Origins of the Efficient Market Hypothesis | "Past Returns Have No Influence on Future Results" | Don't Trust the Charts | Three Forms of the Efficient Market Hypothesis: Weak, Semi-Strong, and Strong | Statistical Testing of the Efficient Market Hypothesis | Efficient Markets Meet Irrational Exuberance | Riding the Trend | Why Do We Follow Others? | Overconfidence and the Winner's Curse | Why Bubbles Don't Easily Burst | Prices Are Wrong | Noise Traders | Optical Illusions and Patterns | Cognitive Shortcuts | The Struggle Between Order and Disorder

12.
Global Warming│We Need a Culture of Healthy Skepticism
In search of causality | The greenhouse effect exists | “It’s not as complicated as rocket science” | Three skepticisms | Criticism of forecasters on global warming predictions | What all climatologists agree on | Questioning computer prediction models | Climate science and complexity | How complex should models be? | Three elements of uncertainty in climate predictions | Observation and records | A ‘hot summer’ is coming | Uncertainty is the essence of predictions | What ‘global cooling’ tells us | Prediction and science are closely linked | The inconvenient truth about temperature records | Another reason why uncertainty estimation is essential | “We’re in a street fight” | Between science and politics

13.
Terrorism: What the Pearl Harbor Attack and 9/11 Terrorism Have in Common
There were signals, but we didn't know what they meant | 'The unfamiliar' and 'the improbable' | Was the 9/11 terrorist attack a 'known unknown'? | A massive attack | The mathematics of terrorism: power law distribution | Defining and measuring terrorism | Magnitude 9 terrorism | 'Thinking big' about terrorism | Why they don't attack shopping malls | Israel's strategy for preventing terrorism | How to read terrorist signals

|Going Out|How is Prediction Possible?
Probabilistic Thinking: Becoming Familiar with Bayesianism | Recognize Your Starting Points and Reduce Your Bias | Try and Make Mistakes | Recognize Predictability
Translator's Note: A New Perspective on the World
|Notes|

Detailed image
Detailed Image 1

Into the book
Given all this, the coronavirus outbreak was less a failure of prediction and more a failure of expert guidance and action.
All the details were wrong and there was a lot of uncertainty, but above all, the overall direction itself was wrong.

Much of this book deals with why experts' predictions are so often wrong and how we can make them more reliable.
But even after the first edition of this book was published in 2012, experts continued to predict world-changing events with a high probability, yet the number of cases where these predictions were largely ignored or misunderstood continued to increase.

In other words, making good predictions and convincing people to take them seriously are two separate issues.
This is especially true if the probability is less than 50 percent, but significantly higher than 0 percent.
A more recent example would be the possibility of Trump being elected president in 2016.
The probability was 28.6 percent.

--- Preface to the Revised Edition│Two Proposals for Better Probabilistic Thinking

This book is about the advancement of information, technology, and science.
It is also a book about competition, markets, and the evolution of ideas.
This book is about how to make us smarter than computers and the mistakes humans make.
It's also a book about how we approach this world step by step, and why we sometimes have to take steps back along the way.

This book deals with 'prediction', which lies at the intersection of all these.
How wonderful it would be if we could have a little more insight than we do now, so that we could make a little fewer mistakes.
This book ponders these questions.
--- Introduction│Signal and Noise

Tetlock found that while hedgehogs on the left and right made particularly poor predictions, foxes, regardless of whether they identified as left or right, made better predictions than hedgehogs who identified as centrist.
The fox may have strong beliefs about how the world should be.
But usually we can separate that belief from an analysis of how the world is and how it will be in the near future.
In contrast, hedgehogs have greater difficulty distinguishing their fundamental interests from their analysis.
To borrow Tetlock's expression, the empty talker "creates a vague fusion out of a tangle of facts and ideological values."
Hedgehogs apply their own biases to the evidence, so they see what they want to see rather than what is actually there.
--- 02.
Politics│My Secret to Predicting Election Results

The epidemiologists I met, unlike forecasters in other fields, were very clear about the limitations of the forecasting models they were running.
For example, Marc Lipsitch added, referring to the flu pandemics of 1918, 1957, and 1968:
"Isn't it fundamentally foolish to make predictions based on three measurement points? Perhaps what we can do is prepare multiple scenarios and develop countermeasures for them."
When experts pretend to be able to make accurate predictions even though they know they can't, many people are put at risk.
Epidemiologists and many medical professionals will be well aware of this fact.
This is a situation where the Hippocratic Oath, which says, "Do no harm to others," is absolutely necessary.
Much of the most thoughtful statistical work I've found comes from the medical field.
Perhaps because the medical field is a place where life and death are at stake, doctors tend to be very cautious.
In the medical field, a poor guess can lead to death.
This means that the consequences can be enormous.

--- 07.
Epidemics│All models are wrong, but some are useful.

One of the nice things about the Bayesian perspective is that it explains very well how we react to changes in the world, while explicitly acknowledging that we have prior beliefs that influence how we interpret new evidence.
(...)
This is not to claim that all empirical beliefs are equally correct or valid.
I believe that we humans can never have perfect objectivity, rationality, or accuracy in our beliefs.
We just try to be less subjective, less irrational, and less wrong.
Making predictions based on your own beliefs is the best (and perhaps only) way to verify yourself.
If objectivity is the pursuit of a greater truth beyond our subjectivity, and if prediction is the best way to see how close we are to that truth, then the most objective among us will be the ones who make the most accurate predictions.

--- 08.
Bayes' Theorem│How to Make Less Mistakes Little by Little

There is one more point that can be helpful when comparing terrorism and earthquakes.
The Gutenberg-Richter law explains that in the long run, the frequency of earthquakes decreases by a factor of 10 for each increase in earthquake magnitude.
However, the energy released in an earthquake is a magnitude function, so it increases exponentially.
(...) What we should mainly worry about are large-scale earthquakes.
Even though such earthquakes occur very rarely.
Now, let's think about terrorism again.
The 9/11 terrorist attacks alone resulted in more deaths (2,977 people, excluding terrorists) than the total number of deaths from all terrorist attacks in NATO member countries over the 30-year period from 1979 to 2009 (Figure 13-7).
If even one terrorist attack using nuclear or biological weapons were to occur, the death toll would be enormous, incomparable to the total number of deaths from the 9/11 attacks.

Such terrorist attacks (even if very unlikely) represent a significant portion of the overall risk.
(...) When it comes to terrorism, we need to think big about the likelihood of a large-scale incident occurring and how we can reduce that likelihood, even if only a little.
This means that we need to focus on signals that point to a large-scale attack and assign strategic priorities to them.
--- 13.
Terrorism: What the Pearl Harbor Attack and 9/11 Terrorism Have in Common

Publisher's Review
Politics, economics, sports, climate, war, terrorism, epidemics, gambling…
Find meaningful signals in a noisy world.


From coping with economic crises to securing our capacity to eradicate disease, everything now depends on how accurately we can predict the future—in other words, how much we can improve the quality of those predictions.
As life accelerates and events become more unexpected, we need to plan and strategize more frequently and more quickly than ever before.
Nate Silver travels from the National Weather Service to Major League Baseball stadiums, from poker tables to the stock market, from the Capitol to NBA stadiums, exploring both successful and unsuccessful forecasts to uncover the common threads among those who make "better"—that is, more accurate—forecasts.
To write this book, I interviewed leaders who shape America, including Nobel Prize winner George Akerlof in economics, former Treasury Secretary Larry Summers, and former Secretary of Defense Donald Rumsfeld. I also conducted in-depth interviews with experts such as professional gamblers, scouts, weather forecasters, and infectious disease experts, and I thoroughly researched and studied related academic papers and articles (please refer to the extensive footnotes).


The COVID-19 Pandemic: What Did We Miss in the 2016 US Presidential Election?
Can probabilistic thinking solve 21st-century challenges like global warming and climate change?
What prediction skills can we learn from the pro sports and poker prediction masters?
Why do bubbles in the stock and real estate markets persist so easily?

Even if the innovations of pioneers in each field are minimal, we can learn a lot from the methodologies they employed.
How can we learn to think probabilistically? Can we apply the insights of an 18th-century Englishman named Bayes to address 21st-century challenges like global warming and extreme weather? Will knowing more about the future help us make wiser decisions in the present?

The Science and Technology of Predicting Moving Targets

"So many predictions fail, so why do some hit the mark?" In this era of "big data," where massive amounts of data are generated daily, why do so many predictions go so far wrong? In the "big data era," humanity is confronted with a vast amount of data, but it also brings with it a staggering amount of useless "noise."
The key to data analysis, as discussed in "Signal and Noise: The Secret of Predicting the Future in an Age of Uncertainty," is effectively removing this "noise" to find meaningful "signals."
The book is broadly divided into two parts.
In Parts 1 and 2, we diagnose the problems associated with prediction, and in Parts 3 and 4, we apply and explore Bayesian solutions.


Part 1 covers the failures of predictions surrounding the recent financial crisis, the success of predictions in baseball, and the field of politics, where predictions can be successful or unsuccessful depending on the approach.
We delve into some of the most fundamental questions inherent in ‘prediction’.

Part 2 examines several dynamic systems, one by one, surrounding climate change, earthquake prediction, economic trends, and the spread of infectious diseases.
Although renowned scientists have been studying these systems for a long time, predictions are even more difficult and tricky because the systems themselves are dynamic.
Therefore, predictions in these fields cannot always be accurate.

Part 3 focuses on solutions using statistical probability techniques.
One of the most important tools Nate Silver uses to distinguish between signal and noise is Bayes's theorem in statistics.
This is a method of improving the posterior probability by deriving a prior probability and then selecting and applying the most probable information when new information becomes available.
Following the introduction of a professional sports gambler who uses Bayes' theorem to achieve great success, we cover the games of chess and poker.
Sports and games serve as excellent laboratories for testing our various predictive skills, as they follow clear rules.
These fields will allow us to more fully understand randomness and uncertainty, and also provide insight into how information is transformed into knowledge.

Part 4 applies Bayes' theorem to more existential problems, addressing global warming, terrorism, and financial market bubbles.
These are not only threatening challenges for society as a whole, but also challenging for forecasting experts.
It's not like there's no solution at all.
If we are willing to face these challenges, our countries, our economies, and our planet can become much safer places.


This book showcases Nate Silver's forecasting philosophy and techniques, while also providing deep reflection on how to embrace and respond to an uncertain future.
This book will be a useful guide for opinion leaders who constantly monitor every factor driving tectonic shifts in the world, entrepreneurs who must constantly anticipate market and consumer desires, readers with a keen interest in social change, and anyone interested in the realities of the "big data" era and how to "survive" in it.


A key insight that changed the paradigm of future prediction.

Silver says that people who embrace knowledge from various fields and try various things, are self-critical enough to admit mistakes, tolerate complex situations and information well, are cautious, and value practical observation over theory, make 'better' predictions than those who pursue big ideas while ignoring details while digging into one field.


Predicting the future isn't done by magic.
The author argues that we should first carefully predict using available information, common sense, and common conclusions drawn from experience by most people, and then slowly gather new information, however small, while acknowledging the incompleteness of that conclusion.
Next, we need to distinguish whether it is a signal containing truth or meaningless noise and evaluate the truth of the prediction based on this.
It takes humility to acknowledge the unpredictable, courage to predict the predictable, and wisdom to know the difference.


Sometimes what matters in forecasting is not absolute accuracy, but 'relative accuracy within the competition'.
Competition is essential for better forecasting.
However, if the forecaster pursues the wrong goal, the prediction may be off track due to competition or even lead to disastrous results.
Forecasters make more accurate predictions when they recognize that they are still in their infancy and that forecasting is a risky science.
The motivation of forecasters is not political, but the pursuit of truth, and they know countless details that will bring them closer to the truth.
Only with this attitude can a forecaster separate the signal from the noise.
GOODS SPECIFICS
- Date of issue: January 5, 2021
- Format: Hardcover book binding method guide
- Page count, weight, size: 824 pages | 1,294g | Checking size
- ISBN13: 9791165213862
- ISBN10: 1165213869

You may also like

카테고리