Part 7: The Project
As this possibility materializes, the perception is growing that the development of superintelligence cannot be left exclusively to startups or private companies. Thus, “The Project” emerges, an initiative that, according to the original document, is comparable to the Manhattan Project in terms of strategic and national security importance.
“The Project” and its inevitability
«The Project refers to a government initiative, similar to the Manhattan Project, but focused on the development of Artificial General Intelligence (AGI) and, eventually, superintelligence. It is considered inevitable due to the growing awareness that AGI and superintelligence represent a technology of critical importance to national security, comparable to nuclear weapons.»
This argument highlights the unavoidable nature of large-scale state intervention. History shows that technologies with transformative and destructive potential have always been subject to government regulation and control. The parallel with the Manhattan Project is particularly telling: just as nuclear fusion changed geopolitics, superintelligence could redefine global power and national security.
Summary of the original text:
The race to AGI is intensifying, and the US government will get involved, culminating in a government AGI project around 2027/28.
It is considered a “far-fetched proposition” that the US government would allow a startup to develop superintelligence. As an analogy, it is difficult to imagine that the US would have developed the atomic bomb and allowed Uber to improvise.
Superintelligence, with its ability to develop new weapons and generate economic growth, will be the focus of international competition. A head start of months could be decisive in a military conflict.
“As so often before—Covid, WWII—it will seem as if the United States is asleep at the wheel, then suddenly the government will shift gears in the most extraordinary way.”
The inability of startups to lead the development of Superintelligence
“Startups, while agile at innovation, lack the infrastructure, resources and focus on national security required for a project of this magnitude.”
The startup business model is driven by speed and cost optimization, factors that conflict with the need for security and control in developing a potentially dangerous technology. The possibility of a startup developing an AGI and deploying it without the necessary safeguards could have catastrophic consequences. Furthermore, the risk of advanced technologies falling into the hands of malicious actors is too great to rely on private companies with commercial incentives.
Forms of US government intervention
«The US government is expected to get involved through a combination of measures, which could include: forming an alliance with major technology companies and AI labs through defense contracts or joint ventures, massive funding of computing infrastructure (such as GPU clusters), establishing a clear and secure chain of command, and intensive collaboration with the US intelligence community.»
The proposed strategy does not suggest a complete nationalisation of AI research, but rather a model of collaboration similar to that in place with companies such as Boeing or Lockheed Martin. This type of relationship would ensure that the government maintains a level of oversight and control without stifling innovation.
Factors that will trigger “The Project”
«Several factors are expected to trigger the launch of “The Project”: significant advances in AI capabilities (surpassing the problem-solving ability of PhDs), huge profits at tech companies due to AI, terrifying demonstrations of AI power (such as amateurs creating biological weapons or hacking critical systems), and growing awareness of Chinese infiltration of American AI labs.»
The original paper suggests that the confluence of these factors will make government intervention inevitable. As AI capabilities approach a critical threshold, governments will need to act to prevent this technology from falling into the wrong hands or developing out of control.
Security and control of the Superintelligence
«Securing and controlling superintelligence will require rigorous measures, such as physical security of data centers, selection and oversight of key researchers, and a clear and accountable chain of command.»
The development of AI and superintelligence is not only a technological challenge, but also a security issue. If an advanced AI system were to become accessible to hostile groups or rogue states, it could become a tool of mass destruction. For this reason, government oversight will need to include extreme measures such as restricting the mobility of key researchers and implementing continuous monitoring systems.
Summary of key points from the original document:
National Defense: Superintelligence will be the most important U.S. national defense project. Existing military capability will become obsolete. A complete retooling of U.S. forces will be needed.
Chain of Command: Superintelligence should not be under the control of private CEOs. “In the world of private labs developing superintelligence, it is quite plausible that individual CEOs have the power to literally stage a coup against the United States government .” A government chain of command and control similar to that of nuclear weapons is needed.
«We have decided that democratic governments should control the military; superintelligence will be, at least initially, the most powerful military weapon. The radical proposal is not the Project; the radical proposal is to bet on private AI CEOs who wield military power and become benevolent dictators.»
Security: Superintelligence is necessary, as private laboratories are unable to guarantee the necessary security against foreign espionage and the proliferation of this technology to rogue states. The leak of AGI laboratories by China is expected to be an important factor in the launch of the project.
Civil Applications of Superintelligence
«While the development of superintelligence will initially be dominated by national security needs, civilian applications are expected to flourish once the initial period of danger has passed and the world has stabilized.»
Historically, many technologies born from military projects have found commercial applications and have been beneficial to humanity. Superintelligence will be no exception: once the critical phase of its development has been overcome, it could transform industries such as medicine, energy and industrial automation.
International stability and superintelligence
“The Project is expected to have as its primary objective the stabilization of the volatile international landscape that will follow the development of superintelligence.”
The geopolitical impact of superintelligence could be comparable to that of the nuclear revolution. The original paper suggests that US leadership in this technology will be crucial to establishing international norms that prevent uncontrolled proliferation and ensure responsible use.
The main argument of The Project is that superintelligence is too important a technology to be left to private companies without government oversight. History shows that when a technology fundamentally alters the global balance of power, state intervention is inevitable. In this sense, the comparison with the Manhattan Project is not exaggerated. The big question is: how can we ensure that this intervention is responsible and does not become an oppressive technological monopoly? The future of superintelligence will depend on how we respond to this question.