In the near future, virtually all software in the world will operate in some form, interfacing with AI. Just as all software running on a PC runs on an operating system, this means that in the future, virtually all software will be integrated with AI.
By the end of 2025, there will likely be more software that integrates with AI than one that doesn't.
--- p.26

There is another special reason for creating humanoids.
This type of humanoid robot is called 'embodied AI'.
What does "having a body" mean? Some AI scientists argue that for AI to truly be "intelligent," it must possess a "body."
That's how you can have a model of the world.
--- p.44

Current large-scale language models cannot eliminate hallucinations.
Because it's like two sides of a coin.
Of course, compared to ChatGPT, hallucination in GPT-4 Turbo is noticeably reduced.
It continues to decrease.
But I don't think it can be eliminated.
--- p.102

This open source culture can be said to be the foundation for the IT industry's ever-accelerating development.
Because once someone writes the code to properly display a chart, everyone on the planet can just use it without having to write it again.
Transformers, Attention, and Rama2, as mentioned earlier, are all open source. In that sense, the IT industry can be considered a melting pot of innovation, where "collective intelligence" operates on a daily basis.

--- p.179

The open source movement is also gaining momentum in AI.
Meta released Rama 3.1 on July 23, 2024 (local time in the US).
Rama3.1 is the largest open-source AI model ever created, with 405 billion parameters.
Meta claimed that this model outperforms OpenAI's GPT-4o and Anthropic's Claude 3.5 Sonnet. GPT-4 is known to have around 1.8 trillion parameters.
It delivers similar performance in less than a third of the size.

--- p.181

The trend toward miniaturization is also gaining momentum.
OpenAI, Microsoft, Google, Meta, Apple, and others are releasing smaller models that rival the performance of larger models that came before.
Miniaturization is inevitable for several reasons.
First of all, current AI uses too many resources.
It is said that it cost 3.7 trillion won to train ChatGPT.
Rama 3.1 ran 16,000 units of the latest GPU H100.
It uses enough electricity to power thousands of households at a time.
It's hard to balance the numbers like this.

--- p.196

OpenAI has not released any specifications or models since GPT-4.
We do not disclose the size of the model, the amount of hardware invested, the dataset used for training, or the training method.
We only exposed the API.
OpenAI cited "corporate confidentiality" as the reason for not revealing this further, but in fact, this attitude seems quite strange considering the founding principles of OpenAI.
“If AI breaks down, something different might need to be done,” said Sam Altman, CEO of OpenAI.
“That’s why no single company should own this type of AI,” he said.
Because of this, OpenAI is being criticized for being essentially a 'closed' AI.
--- p.239

Asilomar becomes the setting for another gathering six years later.
In March 2023, a group of AI and IT experts, including Stuart Russell, Elon Musk (Tesla founder), and Steve Wozniak (Apple co-founder), issued a statement calling for a "stop of large-scale AI experiments."
They said that AI development is spiraling out of control, but without the proper level of planning and management, and that decisions like this should not be left solely to unelected technology leaders.

--- p.283

The debate surrounding AI legislation demonstrates that while the industry sometimes argues for regulation of large models and even announces its own commitments, it is reluctant to actually shoulder the burden.
It is also possible to speculate that, contrary to the claims of the opposition, they are not aware of the possibility of actual danger occurring.
As Professor Hinton and others point out in their letter, “if these risks were truly science fiction, then companies should have no problem taking responsibility for mitigating them.”

--- p.347

The problem is that they are in charge of developing major artificial intelligence projects around the world.
So to speak, we could say that we are being forced to adopt their ideas without even knowing it.
The 'unelected super elite' may not only monopolize the development of artificial intelligence, but also monopolize ideology.
This is another reason why establishing international regulations and norms for the development of artificial intelligence is so urgent and important.
--- p.363

The reason the United States, Germany, and the United Kingdom have become advanced countries and can attract talent from around the world is because they have world-class universities and research institutes.
We now have less than half of our universities with even one basic science department.
It is taken for granted that talented doctoral students do not advance to domestic universities but rather study abroad.
It's sadly shabby compared to how developed countries use their world-class universities to attract talent from around the world like magnets.
If the government fails to change the pace of its science and technology policies and abandons the attitude and strategies of a latecomer, we may soon find ourselves back in the dark as a developing country.

--- pp.395-396