The Unfinished Story: The Fable of the Sparrow and the Owl
introduction
Acknowledgements
1 Past development and present capabilities
The development of history and the great history _ Great expectations _ Times of expectation and despair _ Cutting-edge technology of artificial intelligence _ Some opinions on the future of machine intelligence
2 Some Paths to Superintelligence
Artificial Intelligence _ Whole Brain Emulation _ Biological Cognitive Ability _ Brain-Computer Interface _ Networks and Organization _ Summary
3 Forms of Superintelligence
Velocity Superintelligence _ Collective Superintelligence _ Qualitative Superintelligence _ Direct and Indirect Influence _ The Source of the Advantages of Digital Intelligence
4. Dynamics of Intelligence Diffusion
Timing and speed of leap _ Resistance _ Optimization ability and large-scale diffusion _
5 Clear Strategic Advantages
Will the leader have a clear strategic advantage? How large will a successful R&D project be? From a clear strategic advantage to monopolistic dominance.
6 Cognitive Superpowers
Functionality and Superpowers _ Scenarios for AI to Take Control _ The Power to Overcome Natural Resistance and Resistance from Other Agents
7 Superintelligent Will
The Relationship Between Intelligence and Motivation - Instrumental Convergence
8 Is the intended ending doomed?
Is the predicted end of the intelligence proliferation an existential catastrophe? _ A dangerous turn _ Malignant failures
9 Control Issues
Two-Agent Problem _ Ability Control Method _ Motive Selection Method _ Overview
10 Oracle, Genie, Sovereign, Tool
Oracle _ Genie and Sovereign _ Tool-AI _ Comparison
11 Multipolar Scenarios
Horses and Humans _ Life in the Algorithmic Economy _ The Emergence of a Transitional-Post-Monopolistic Dominance System?
12 Value Acquisition
Value-loading problem _ Evolutionary selection _ Reinforcement learning _ Combined value assignment _ Motivational ad hoc hypothesis _ Value learning _ Emulation coordination _ Institutional design _ Overview
13 Selecting the selection criteria
The need for indirect normativity _ the will to consistently assume _ the moral law model _ act as I wish _ the list of elements _ getting close enough
14 Strategic Drawings
Science and Technology Strategy _ Paths and Facilitators _ Collaboration
15 Decision Time
Philosophy with a time limit _ What should be done? _ Let the best of human nature come out and solve the problem.

main
References
Translator's Note
Search

The 2016 match between AlphaGo and South Korean professional Go player Lee Sedol captured global attention, and the result elicited both surprise and concern.
This match, which clearly showed the current state of artificial intelligence, made everyone realize that the day when artificial intelligence will be used in the real world is not far off.
Humans have achieved a dominant position on Earth with their current intelligence.
So, if a superintelligence far surpassing human intelligence is developed, what kind of future awaits humanity? This book will explore how we might address this issue, which is considered the most important and serious problem facing humanity.
Before a superintelligence smarter than humans is developed, humanity will have only one chance.
How we utilize that opportunity now depends on the efforts of all of humanity.

Chapter 1 of this book describes the history and current developments in the field of artificial intelligence, while Chapter 2 explores several possible technological paths toward superintelligence and cautiously predicts when it might become possible.
Chapter 3 examines the forms of superintelligence (speed superintelligence, qualitative superintelligence, and collective superintelligence) and explores the potential of machine-based intelligence.
Chapter 4 examines when artificial intelligence, having achieved human-level intelligence, will achieve an intelligence explosion and reach superintelligence.
Chapter 5 addresses the development of superintelligence, whether a single superintelligence will be developed or multiple superintelligences will be developed simultaneously.
Chapter 6 explores the various abilities that superintelligence might possess and considers what superintelligence might be able to do with those abilities.
Chapter 7 examines what ultimate goals superintelligence might achieve with its immense capabilities and what motivations might drive those goals.
Chapter 8 takes a fresh look at our current perspective on superintelligence, emphasizing that superintelligence is a completely new intelligence, distinct from humans.
Chapter 9 shifts the focus of the discussion to the problem of controlling superintelligent agents, proposing methods for controlling superintelligence from two perspectives: ability control and motivation selection.
Oracle, Genie, and Sovereign tools, which are programs that can be used for control problems, are covered in Chapter 10.
Chapter 11 explores what happens in a post-transition society, a multipolar ending, where multiple, conflicting superintelligent agents are deployed.
Chapter 12 delves into the motivational selection methods that could fundamentally control superintelligence.
Chapter 13 is about what values ​​we should choose to give to superintelligence.
In Chapters 14 and 15, we will look at the findings from the previous chapters from a macro perspective and consider the big picture that can be gleaned from them.
And finally, I propose several things humanity can do now to avoid the existential threat posed by superintelligence in the future.

This book raises fundamental questions that humanity must address as it moves from artificial intelligence to superintelligence, and looks ahead to what we can do to meet challenges humanity has never experienced before.
Superintelligence, which is expected to be developed within this century, will determine the future of humanity.
The future of the artificial intelligence era, vaguely imagined through science fiction novels and movies, will be reimagined as a vivid, present-day reality through this book.

Before the prospect of a massive proliferation of intelligence appears, humans are like small children playing with bombs.
This illustrates the dissonance between the power of toys and the immaturity of human behavior.
Superintelligence is a daunting goal that we are not ready for now and will not be ready for for some time.
It's almost impossible to predict when an explosion will occur, but if we hold the device close to our ear, we'll be able to hear a faint ticking sound.

The wise thing for a child holding an unexploded bomb to do is to carefully put the bomb down, quietly leave the room, and alert the nearest adult.
But in the problem we are now dealing with, we are dealing with not one child, but many, and each child has an independent trigger.
There is little chance of finding a sensible way to get all the hazards out.
Some foolish guy will press the ignition button to see what happens.

No one will be able to escape safely from the storm of intelligence proliferation that will descend on the entire sky in all directions.
In that case, there will be no visible adult there to report the incident and ask for help.