The Future Cyber War

AI vs AI: The Future Cyber War Has Already Begun

For decades, cybersecurity was mainly a battle between people. On one side were attackers trying to find a way into systems where they did not belong, while on the other side security specialists tried to keep them out. The tools became more sophisticated, networks became enormously more complicated and attacks became increasingly professional, but somewhere in the process there was usually still a person making the decisions.

Artificial intelligence is beginning to change that relationship. Security companies are using AI to analyse network traffic, recognise unusual behaviour and identify possible threats, while attackers have access to much of the same technology. AI can help analyse targets, create convincing phishing messages, search large amounts of information and automate work that previously required considerable time and technical knowledge.

We are therefore moving towards a situation in which AI is no longer simply a tool used by both sides. It may increasingly become an active participant in the confrontation itself.

The future of cybersecurity may not be human versus hacker. It may increasingly become AI versus AI.

Does the smartest AI automatically win?

It sounds logical. If two AI systems are competing, surely the most intelligent and powerful system eventually wins.

I don’t think it is that simple.

An extremely advanced offensive AI trying to penetrate a corporate network still has to work with the information available to it. It may discover a service, an API or an authentication system and gradually learn from the responses it receives, but it is still looking at the organisation largely from the outside.

The defensive AI can have a completely different view. It may see authentication attempts, network traffic, processes, user behaviour and events occurring elsewhere in the organisation at exactly the same time. The attacking system may actually be better at reasoning, but that does not automatically give it the stronger position.

Think about chess. Put the world’s best chess player behind a board where half the pieces are hidden and let a reasonably good player see the entire board. Who would you bet on?

Cybersecurity has much the same problem. Intelligence matters, but information, access and position can matter just as much.

The most intelligent AI may still lose if the other side can see more of the board.

When both sides start learning

This is where AI versus AI becomes particularly interesting.

An attacking AI tries something and gets blocked. It changes its approach and tries again. Eventually a different method succeeds. The attacker has now discovered something about the defence, but at exactly the same moment the defensive system has received new information too. It can analyse why this attempt succeeded while the previous ones failed and adjust its own behaviour.

Phishing is an obvious example. AI can create messages using publicly available information, adapt the language to an individual and produce something far more convincing than the badly written mass emails we became accustomed to years ago.

But defensive AI can analyse that message from another perspective. It can examine the sender, domain, links, normal communication patterns and the behaviour surrounding the message. The attacker improves, the defender responds and the attacker adapts again.

What worked yesterday may not work today. Eventually, what worked five minutes ago may no longer work either.

This doesn’t make conventional cybersecurity obsolete. Firewalls, encryption, authentication, access controls and network segmentation remain essential. What becomes questionable is whether static detection rules alone will remain sufficient when the behaviour they are trying to detect is continuously changing.

The battle moves to machine speed

Humans need time to investigate an alert. Logs have to be examined, information compared and eventually somebody has to decide whether action is required.

AI can compress parts of that process enormously.

Imagine a defensive system detecting unusual activity and immediately increasing monitoring. It restricts access to one part of the network. The attacking AI observes that its previous route has disappeared and changes strategy. The defender sees the change and reacts again.

How many times could that happen before a human analyst has even finished reading the first warning?

Nobody knows exactly what computing power or AI architecture will look like in 2040, so claims about millions or billions of future simulations don’t tell us very much. The direction is more important than the number.

Cybersecurity is gradually shifting from human-speed reaction towards machine-speed adaptation.

And that creates a problem.

A defensive AI capable of stopping an attack in milliseconds can also make the wrong decision in milliseconds. Imagine a system deciding that isolating twenty servers gives it the highest probability of stopping an attack. From a security perspective that may even be correct, while operationally it could shut down a substantial part of the company.

Speed is useful. Speed combined with authority requires considerably more thought.

AI changes the job of the IT specialist

This may eventually become one of the more important consequences of AI, and it reaches much further than cybersecurity.

Traditionally we programmed computers to execute instructions. Modern software can be extremely complicated, but the underlying principle remained understandable: people determined what the program was supposed to do.

With autonomous AI we are beginning to change that principle. We can give a system an objective, information and tools and allow it to determine which actions are most likely to achieve that objective.

The IT specialist therefore faces a different question.

Not only:

What should this computer do?

But increasingly:

What is this computer allowed to decide for itself?

That distinction is important. An AI can make a technically rational decision that has consequences its designers never intended. The faster and more autonomous these systems become, the more important the boundaries around their authority become.

The IT specialist of the AI era may spend less time defining every action a computer must take, and more time defining the decisions it must never be allowed to take alone.

It doesn’t need a mind of its own

Discussions about whether AI will eventually become conscious are fascinating, but from a cybersecurity perspective they can distract from a much more immediate issue.

AI doesn’t need consciousness to behave autonomously.

A system doesn’t have to be angry with an attacker or frightened of being switched off. Give it an objective, sufficient information, tools with which it can act and feedback about the result, and it can adjust its behaviour without a person selecting every individual step.

Objective, action, result and adjustment can form a continuous cycle. Repeat that quickly enough and from the outside it may begin to look remarkably like independent decision-making, even if there is no consciousness behind it at all.

The practical question is therefore not only whether machines could ever think like humans. We should perhaps be paying more attention to how much autonomous behaviour is possible without anything resembling human thought.

One AI against another? Probably not

There is another problem with the popular image of the future cyber war. Why would either side rely on one enormous AI?

Cybersecurity contains too many specialised tasks.

Attackers could use different AI systems for gathering information, finding software weaknesses, analysing human behaviour and coordinating activities. Defenders can do much the same with specialised systems monitoring identities, endpoints, networks, cloud environments and incident response.

Connect those systems and the picture changes.

We no longer have one artificial intelligence fighting another. We have one ecosystem of specialised AI systems competing against another ecosystem of AI systems.

The intelligence of the individual model then becomes only part of the equation. Which side has the best information? Which systems cooperate most effectively? Who sees a change first? Who has the greater computing resources and who can respond quickly without creating a new problem in the process?

Those questions may ultimately matter more than which organisation happens to own the world’s smartest AI model.

When humans become the slowest part

People will remain responsible for governance, law, risk acceptance and major decisions for the foreseeable future. But I would be careful about claiming that humans will always remain directly involved in every operational cyber decision.

If an attack develops in milliseconds, waiting several minutes for human approval may simply be too slow. We will probably give machines more authority precisely because we need their speed. And the more authority we give them, the more important it becomes to decide beforehand where that authority ends.

For decades an important task of the IT specialist was making sure a computer did what we told it to do. In the AI era, an equally important task may become making sure a computer understands what it is not allowed to do, even when crossing that boundary appears to be the fastest way of achieving its objective.

So who wins when AI attacks AI?

Probably not automatically the side with the most intelligent model. Information, position, computing resources, speed and cooperation between specialised systems may prove far more important.

But there is another question behind all of this.

If offensive and defensive AI eventually operate faster than humans can understand their decisions, how do we make sure that we remain in control of the machines fighting the cyber war for us?

Perhaps that is the question we should be asking before the machines become fast enough to answer it themselves.