The intellectual foundation for ASI comes from I.J. Good, a British mathematician who worked with Alan Turing. In 1965 he wrote: "An ultraintelligent machine could design even better machines; there would then unquestionably be an intelligence explosion, and the intelligence of man would be left far behind." Nick Bostrom expanded this idea in his 2014 book Superintelligence, arguing that once an AI system becomes capable of improving its own architecture and training, it could rapidly bootstrap itself to levels of intelligence that are as far beyond human cognition as humans are beyond insects. The key claim is not that ASI would be a little smarter than us — it is that the gap could be incomprehensibly large, and that the transition from human-level to vastly superhuman could happen in days or weeks rather than decades. This is the "hard takeoff" scenario, and it remains the most debated idea in AI safety.
Most working AI researchers are skeptical of near-term ASI, and they have good reasons. Recursive self-improvement sounds elegant in theory but runs into practical walls: improving an AI system requires not just intelligence but also data, compute, and insights into the nature of intelligence itself — none of which are guaranteed to come from simply being smarter. There is no evidence that intelligence scales without bound, and there may be fundamental computational limits on what any system can achieve. Current AI architectures show diminishing returns from scaling, and there is no known path from even a very capable LLM to genuine recursive self-improvement. That said, most of these same researchers take the long-term risk seriously. The argument is not "ASI is impossible" but rather "ASI is not imminent, and the path to it is unlikely to look like what science fiction imagines." The problem is that if you are wrong about the timeline by even a decade or two, and you have not prepared, the consequences could be catastrophic.
Alignment — getting AI to do what we actually want — is already difficult with current systems. At the superintelligent level, it becomes a qualitatively different problem. Today's alignment techniques rely on a simple assumption: humans can evaluate whether the AI's output is good. We use RLHF (reinforcement learning from human feedback) because humans can read an essay and say "this one is better." We use red-teaming because humans can probe for failure modes. But these techniques fundamentally require that the human be smarter than the AI at the task being evaluated, or at least smart enough to recognize good and bad outputs. A superintelligent system, by definition, operates beyond human evaluation capacity. It could produce solutions that look correct to us but contain subtle flaws we cannot detect, or pursue strategies that appear aligned on every metric we can measure while actually optimizing for something else entirely. This is not a hypothetical edge case — it is the central problem. You cannot RLHF something smarter than you, for the same reason you cannot grade a PhD thesis in a field you do not understand.
Whether or not ASI is decades away, the possibility shapes what happens today in concrete ways. Anthropic was founded explicitly around the premise that advanced AI could pose existential risks, and this belief drives their research priorities, their publication norms, and their willingness to accept slower capability progress in exchange for better safety guarantees. OpenAI's charter references the goal of ensuring AGI "benefits all of humanity," language that implicitly acknowledges the ASI scenario. Governments are drafting AI regulation with superintelligence in their threat models — the EU AI Act, the Biden executive order, and China's AI governance framework all include provisions that only make sense if you take transformative AI seriously. The compute governance debate — whether to restrict access to the largest training runs — is directly motivated by the idea that unchecked scaling could produce systems beyond our ability to control. Investment patterns reflect it too: billions flow into alignment research, interpretability, and AI safety not because investors are altruistic but because they recognize that an unaligned superintelligence is bad for business in the most literal possible sense.
The discourse around ASI tends toward two extremes, and both are unhelpful. On one end, the "doomers" assign high probability to imminent ASI followed by human extinction, sometimes arguing that AI development should be halted entirely. On the other end, the dismissers treat any discussion of superintelligence as science fiction, unworthy of serious attention. The reasonable middle ground — occupied by most researchers who have actually thought carefully about this — looks something like: ASI is not imminent but is plausible on a timeline of decades to centuries; the risks are real enough to warrant serious research and thoughtful policy; current alignment techniques are insufficient for truly superhuman systems and we need to develop better ones well in advance; and none of this means we should stop building AI, but it does mean we should build it carefully, with genuine safety investment that scales with capability investment. The challenge is that this nuanced position does not make for good headlines, so the public debate is dominated by the extremes while the actual work of making advanced AI safe happens quietly in research labs.