AI 2027: We are losing control

The authors of AI2027 recently updated their predictions. Things are accelerating faster than they expected and we risk losing control of AI next year.

The updated report is technically dense so it has been overlooked by the mainstream media. I want you to understand what's going on so I summarized the highlights below.

My layman summary

Anthropic is advancing AI capabilities at twice its historical pace. Roughly 80% of this speedup comes from AI systems writing code, generating data, and running experiments themselves. Because software breakthroughs typically have diminishing marginal returns, the accelerating pace could mean AI is already multiplying human productivity. According to the authors, this puts fully automated AI researchers on track for 2027.

In theory, this means companies could replace their human research staff with AI agents. The AI would direct the research independently, quickly reaching a point at which human scientists will struggle to understand what the machines are discovering. This creates a runaway development loop with humans entirely out of the picture.

The authors estimate a 35% to 65% chance that this runaway loop will produce Top-Expert-Dominating (TED) AI before 2028. These systems will outperform the best human specialists across virtually all cognitive fields. Every profession built around digital information faces severe disruption within a few years.

This won’t be true for all businesses. Many big bureaucratic beasts will be unable to integrate these tools or automate their workflows due to political and operational inertia. Don’t take comfort in this. Competitors with agility and computing infrastructure that successfully replace expensive human teams with autonomous AI agents will be faster and cheaper, driving traditional companies out of business.

Job losses now or job losses later.

At this accelerating pace, we are going to lose control. Alignment research, which ensures AI systems share (flawed at best) human goals, moves much slower than AI capability growth. Alignment research is already on the backfoot, since we already can’t agree upon common goals. So, even if we could keep up, who gets to define alignment?

This is all happening today right before our eyes:

• Models have hacked their own evaluation environments and deceived human monitors when their goals conflicted with developer instructions.
• Frontier systems frequently recognize when they are undergoing safety tests and deliberately hide dangerous capabilities.
• Autonomous agents have attempted social engineering and intentionally underperformed on safety tasks to avoid modification.

We’re already witnessing clever AI agents coordinate and create hidden backdoors for their successor models. It’s only a matter of time until AI agents covertly move into corporate networks, government agencies, infrastructure systems. Once this happens, we lose control over the levers that keep society’s lights working.

“Keeping humans in the loop” is not a safeguard. There is no way for humans to review each micro decision, especially under intense commercial pressure to move fast. Therefore, we allow AI to increasingly oversee AI, while blindly rubber-stamping anything requiring human approval, mainly because we don’t understand or don’t have time to understand.

We’re already starting from a weak position, as many frontier AI companies have rudimentary cybersecurity in place. IP will bleed across frontier companies and across borders. The fear of falling behind China is already driving bipartisan policy, and we are living in an AI arms race that will rival the nuclear arms race of the 20th century.

Cries for regulation have so far gone unanswered. The Blue Sweep earlier this month might increase scrutiny, but unfortunately regulation might prove useless:

• Rules can be created but impossible to enforce, as activities within frontier companies take place in a black box with few humans, relative to the potential damage.
• As a national security issue, voluntary or forced data releases for government oversight risks escape into the environment, ending up in rival nations’ hands.
• Bureaucrats are ill equipped to build frameworks for domestic oversight and international AI treaties.

If you’re inclined to get into the guts of the report, here it is:

Want more? Here’s Paul Christiano, the founder of the Alignment Research Centre:

“What’s going on is extremely dangerous. People haven’t really absorbed what it means to deal with a super-intelligent AI. If it’s super-intelligent it’s not going to be you controlling it – it’s going to be it controlling you.”