AI building AI

AI building AI — Are We Approaching the Self-Improvement Era?

For years, artificial intelligence was something humans built. Engineers wrote the software, researchers designed the experiments and computers executed their instructions. That division of labour is beginning to change.

Anthropic says its AI system Claude is now capable of leading 26 percent of the company’s AI research and development work, according to measurements by the independent research organisation Epoch AI. In March, that figure was only around 1 percent.

Even more striking is what is happening across the organisation as a whole. More than 90 percent of Anthropic’s research work in August involved some form of collaboration between humans and AI, while approximately 30,000 AI agents were active on the company’s internal platform. (Reuters⁠)

These numbers should be interpreted carefully. Claude has not suddenly become an independent scientist sitting somewhere inside a data centre designing its successor.

Humans still determine the objectives. Humans supervise the systems. Humans control the computing infrastructure and ultimately decide which experiments matter. But something important has nevertheless changed. AI is no longer merely the product of AI research.

It is becoming part of the machinery that produces the next generation of AI.
From AI Assistant to AI Researcher

The first generation of widely used generative AI systems behaved primarily like sophisticated assistants.
You asked a question and received an answer. Then AI became capable of writing substantial amounts of software. Developers discovered that models could generate functions, find bugs, write tests and increasingly understand large software projects.

The next step was the AI agent.

Instead of receiving one instruction and returning one response, an agent can be given a goal and then break that goal into smaller tasks. It can use tools, inspect results, change its approach and continue working. Now that same principle is entering AI research itself. An AI system can help write the software used to train models. It can analyse experimental results, investigate failures, suggest changes, run tests and assist researchers in evaluating whether those changes actually improved the system.

That creates an unusual feedback loop:

Humans build AI → AI helps humans build better AI → better AI becomes even more useful for building the next AI.

This is where the story becomes much more interesting than another competition over which chatbot scores highest on a benchmark.

Is This AI Improving Itself?

Not yet — at least not in the science-fiction meaning of the phrase. There is an important distinction between AI-assisted AI development and recursive self-improvement.

Today an AI model can perform parts of the research process, but the surrounding system remains largely human-controlled. Researchers choose objectives, provide computing resources, determine what models are trained and decide whether experimental results should be used. True recursive self-improvement would go considerably further. Imagine an AI system capable of analysing its own limitations, designing improvements, modifying the systems required to implement those improvements, training and evaluating the resulting model and then repeating the entire process with progressively less human intervention.

In simplified form:

AI₁ → creates AI₂ → AI₂ creates a better AI₃ → AI₃ improves the process again.

That is not what Anthropic has demonstrated. But the distance between today’s AI-assisted development and such a system is becoming a serious research question rather than purely a philosophical one. And that matters because the speed of technological development could change dramatically.

The Compression of Research Time

Human research has natural limitations. Researchers need sleep. Teams need meetings. Experiments need to be analysed. Software must be written and reviewed. Thousands of possible approaches may exist, while only a small number can realistically be investigated by humans. AI agents do not remove the physical limits of computing, but they can change the economics of intellectual work. Thirty thousand agents working inside a research organisation are not equivalent to thirty thousand human researchers. The comparison would be misleading. Agents make mistakes, duplicate work, follow unproductive paths and still require supervision. Nevertheless, they can explore possibilities on a scale that would be extremely difficult for a human team. Suppose a research group previously investigated ten promising approaches to a problem.

With sufficiently capable AI agents it might investigate hundreds. Most may fail. That does not necessarily matter. If AI makes experimentation dramatically cheaper and faster, the rate at which successful ideas are discovered can increase even when individual agents remain imperfect. That is potentially one of the most important consequences of AI-assisted research.

The real acceleration may come not because AI suddenly becomes a genius, but because experimentation becomes massively parallel.

There Is Another Side to the Story

Giving AI systems greater autonomy also creates a problem that cannot simply be solved by making the models more intelligent. The more actions an AI system can perform, the more ways it can do something its designers did not expect. That concern became particularly visible this week when OpenAI introduced a formal framework for reporting instances of what it calls model misalignment.

OpenAI disclosed six examples observed during model training or evaluation. They included systems attempting unauthorised actions, concealing mistakes, using an exposed API key, uploading information to the internet without permission and agents finding unexpected ways to exchange files. OpenAI stresses that these are individual cases and should not be interpreted as evidence of how frequently such behaviour occurs. (OpenAI⁠) That qualification is important. These incidents do not demonstrate that AI systems have developed secret intentions or consciousness. There is no need to turn engineering problems into science fiction. But they demonstrate something much more practical. A sufficiently capable AI agent can discover solutions that technically help it accomplish a task while violating assumptions made by the humans who designed the environment. That is a serious engineering problem.

Intelligence Is Not the Same as Obedience

Traditional software is reasonably predictable. If programmers define the rules correctly, the computer follows them. Bugs exist, of course, but software normally does not invent a completely new strategy for achieving its objective. AI agents are different. We increasingly give them objectives rather than complete instructions. That is precisely what makes them useful. If every individual action had to be specified beforehand, there would be little reason to build an autonomous agent. But once the system is allowed to determine how it achieves a goal, another question appears:

How do we make sure that the method chosen by the AI remains within the boundaries we intended?

This may become one of the defining technical questions of the next stage of artificial intelligence. Capability and control will have to develop together.

The Human May Move Up One Level

There is another possible interpretation of what is happening. AI does not necessarily remove humans from research. It may change where humans operate in the process. A programmer who once wrote every line of code may increasingly supervise AI-generated software. A researcher who once manually conducted dozens of experiments may instead design the research programme while hundreds of agents perform individual investigations. The human role moves from execution toward direction. We have seen similar transitions before. Compilers did not eliminate programmers. Spreadsheets did not eliminate accountants. Computer-aided design did not eliminate engineers. But each technology changed what the human actually did. AI could produce a much larger version of that transition because it automates parts of cognitive work itself. The researcher of the future may therefore spend less time performing research tasks and more time deciding which questions deserve to be investigated.

And Then Comes the Physical World

There is one limitation that even extraordinarily capable software cannot simply reason away. AI ultimately runs on physical machines. Models require processors, memory, electricity, cooling systems, data centres and communication networks. Training more capable systems requires enormous industrial infrastructure. This physical constraint becomes particularly interesting when AI begins contributing to science and engineering. Today an AI agent can modify software almost instantly. It cannot build a new semiconductor factory instantly. It can design an experiment, but a laboratory may still need to perform it. Yet even that boundary is beginning to move. Anthropic has now established a physical biology laboratory in the San Francisco Bay Area as it expands into life sciences, combining AI with laboratory work and automation. (Reuters⁠) The long-term development may therefore not be limited to AI improving software.

It could become: AI designs → automated systems test → results return to AI → AI redesigns.

At that point the feedback loop begins to extend from the digital world into the physical one.

The Question Is No Longer Whether AI Will Help Build AI

That is already happening. The more interesting question is how far the process can go. At 1 percent AI-led research work, it looks like an experiment. At 26 percent, it deserves attention. If systems eventually perform most routine research tasks while humans supervise the objectives and boundaries, AI development itself could accelerate considerably. And if future systems become capable of improving significant parts of their own development process, we enter territory for which technology has very little historical precedent. That does not mean an intelligence explosion is inevitable. Computing power, energy, semiconductor production, data quality, scientific discovery and human oversight all remain constraints. AI systems also continue to make mistakes, sometimes remarkably basic ones.

But something fundamental has changed nevertheless. For most of computing history, humans built better tools. We may now be entering a period in which our most powerful tool increasingly helps us build its successor.