Smartest AI System May Not Win

When AI Attacks AI: Why the Smartest System May Not Win the Cybersecurity War

Smartest AI System May Not Win. Artificial intelligence is rapidly finding its way into cybersecurity. It is being used to detect suspicious behaviour, analyse enormous amounts of data and identify threats much faster than humans could ever hope to do. But the same technology is, of course, also available to the other side.

That raises an interesting question.

What happens when AI is no longer mainly fighting human hackers, but another AI? The obvious answer seems to be that the smartest and most powerful AI will win. After all, if one system can reason faster, analyse more possibilities and develop better strategies than the other, the outcome should be fairly predictable.

But is it? ………Probably not.

Cybersecurity has never been determined by intelligence alone. Information, access, computing power, speed and, perhaps most importantly, where you are positioned in the system can be just as decisive. Think about two computers playing chess. One is vastly more powerful than the other, but it is allowed to see only half of the board. The weaker computer can see every piece and every move. Which one would you bet on? That is roughly the problem we encounter when we start thinking seriously about AI fighting AI.

The attacker doesn’t see everything

An offensive AI looking at a corporate network from the outside does not magically know what is inside. It has to find out. It may discover that a service responds on a particular port. It finds an API. Authentication is required. A certain request produces an unusual response. Nothing spectacular has happened yet, but every response tells the system something it did not know before.

So it tries something else.

And then something else.

In simple terms, it is observing, analysing, testing and adapting. What failed a minute ago becomes information for the next attempt. That is already different from the rather static picture we used to have of a computer attack. An AI does not necessarily have to repeat exactly the same procedure. It can use the result of an unsuccessful attempt to decide what to do next.

But there is someone on the other side of the door.

The defensive AI may be watching the same activity from a completely different position. It can potentially see authentication attempts, network traffic, processes running on machines, unusual user behaviour and thousands of other events occurring elsewhere in the organisation.

The attacker sees the door.

The defender may be able to see much of the building.

That difference matters.

Suppose a server normally receives one hundred requests an hour and suddenly receives five thousand. That doesn’t prove an attack is taking place. Perhaps there is a perfectly innocent explanation. But combine it with an unusual login attempt, unexpected process activity and traffic from somewhere the organisation has never seen before and the picture changes. A defensive platform receiving that information can increase monitoring, restrict traffic, challenge credentials or isolate a system.

It doesn’t necessarily have to be more intelligent than the attacking AI.

It may simply know more.

Both sides start predicting each other

This is where AI cybersecurity becomes particularly interesting. The attacking AI isn’t really fighting the defensive AI directly. It is interacting with a network that the defensive system is trying to protect. After enough attempts, however, something else starts happening.

The attacker begins to learn what gets detected.

The defender begins to learn what the attacker is trying.

At that point both systems are no longer dealing only with computers and networks. They are effectively trying to predict each other’s behaviour. The offensive system wants to know what the defence is likely to recognise and how it will respond. The defensive system wants to know what the attacker is attempting to achieve and, preferably, what it will try next.

That begins to look less like traditional hacking and rather more like a strategic game.

There is one important difference, though. A chess game eventually ends.

Cybersecurity doesn’t.

Consider something as ordinary as phishing

An AI generates a phishing message and sends it.

Blocked.
It changes the wording and tries another variation.
Blocked again.

The third version is constructed differently and this time it gets through. The attacker has now learned something. Unfortunately for the attacker, so has the defender. The defensive system can examine what was different about the successful message and combine that information with other signals. What worked five minutes ago may no longer work on the next attempt.

The offensive system adapts again…….And so does the defence.

This doesn’t mean that conventional cybersecurity suddenly becomes useless. Firewalls, authentication, network segmentation, access controls, encryption and the principle of giving systems no more authority than necessary remain essential. What may become increasingly inadequate is relying on static detection rules alone when the behaviour you are trying to detect is itself continuously adapting.

That is a fundamentally different problem.

So which AI is actually the most powerful?

This is where I think we need to be careful with the word intelligence.

Imagine an extraordinarily capable future AI with relatively little computing power, incomplete information and only a narrow view of the network it wants to penetrate. Now put a somewhat less capable AI on the other side, connected to a huge security infrastructure. It receives continuous telemetry from the network and endpoints, knows which users should be where, can correlate events across the organisation and has substantial computing resources available.

The first AI may be considerably better at reasoning. I would still hesitate to bet on it.

Speed matters. Coverage matters. Information matters. Computing resources matter. Position matters.

But we should not make the opposite mistake either and assume that the defender automatically has the advantage. A defender has to protect an enormous environment containing software, people, devices, cloud services, suppliers and countless connections. An attacker may need to find only one useful weakness. That asymmetry has existed in cybersecurity for decades. AI doesn’t remove it. It may actually make it more important.

Then we add speed

Humans are still deeply involved in cybersecurity today. Analysts investigate warnings, inspect logs, compare events and decide whether action is necessary.

That takes time.

AI can reduce that time dramatically. We can imagine future offensive systems evaluating enormous numbers of possible approaches in simulation before doing anything to the actual target. At exactly the same time, defensive systems could be modelling attacks and testing possible responses.

Whether that means thousands, millions or eventually billions of simulated possibilities isn’t really the important question. Nobody can sensibly predict the computing environment of 2040 with that degree of precision.

The direction is much more interesting than the number. Cybersecurity is moving from human-speed reaction towards machine-speed adaptation.

And there is a problem hidden in that development.

If a defensive AI can identify and stop an attack in milliseconds, that is fantastic.

If it makes the wrong decision in milliseconds, it can cause damage equally quickly.

A company cannot simply give an AI enormous authority because it happens to be fast. The faster and more autonomous these systems become, the more important their boundaries become. That may eventually become one of the biggest jobs in IT: not telling a computer exactly what to do, but determining what an autonomous computer is allowed to do.

It probably won’t be one AI against another anyway

There is another reason why I don’t believe the future cybersecurity battle will resemble two electronic superbrains fighting each other.

Why would we build it that way? Cybersecurity contains too many different specialisms. An attacking organisation could use one AI to examine externally visible systems, another to look for software weaknesses, another to analyse human behaviour and yet another to coordinate information from the others. The defender can do exactly the same thing. Identity protection, network monitoring, endpoint security, fraud detection and incident response can all have specialised AI systems.

Now connect them.

Suddenly we don’t have one AI attacking another AI.

We have one ecosystem of AI systems competing with another ecosystem of AI systems.

That is a much more interesting prospect because the quality of the individual AI is then only part of the equation. How well do they communicate? What information can they access? Can one system recognise something and warn the others quickly enough? Who is allowed to make a decision? And what happens when two defensive systems disagree?

Those may sound like organisational questions. They are. We are simply starting to ask them about machines.

And humans are still sitting somewhere in the middle

This is perhaps the part that concerns me most.

Imagine an offensive AI changes its behaviour. A defensive AI notices it and changes a security control. The offensive system observes the result and selects another strategy. Another defensive system detects that change and responds again. By the time a human analyst has opened the first warning, both sides may already have changed strategy several times. We don’t need science fiction or a conscious computer for that to happen. None of these systems has to want anything in the human meaning of the word.

They need an objective, information, computing power, the ability to act and enough freedom to adapt. That is sufficient to create behaviour that can become extremely difficult for a human being to follow in real time. So perhaps we have been asking the wrong question. The future cybersecurity contest may not be decided by which side owns the smartest AI. It may be decided by which side has the best information, the strongest position, the fastest feedback and the best-coordinated collection of specialised AI systems.

And then there is one final question that we should probably ask before giving all those systems more autonomy:

Can the humans still understand what is happening quickly enough to remain in control?

We may discover that this question is considerably more important than determining which AI is the smartest.

And it is probably much closer to the cybersecurity future we are actually building.