AI played wargames. It wasn't reassuring. (Patreon)

[This is a transcript with links to references.]

A group of researchers had artificial intelligence play wargames, and that gives us a good idea for how we could all die. Let’s have a look.

Last year in June, the United States’ Department of Defense released its new strategy for the adoption of artificial intelligence. According to the document, the main reason for using AI in the military is a “decision advantage” that among other things allows  “Fast, precise and resilient kill chains.”

The DoD, and doubtlessly most,  if not all, other defence organizations are cooking up their own AI and feed them will all kinds of data we’d be shocked to hear even exists. But for the time being many of them probably make do with what’s available, and yes that means they probably use the same AIs as everyone else, Large Language models like GPT.

Indeed, according to a Bloomberg article, the DoD has conducted a set of tests last year in which they evaluated the use of five different large language models in conflict situations. They quote US Air Force Colonel Matthew Strohmeyer with saying that they fed the language models with secret-level data , that the test was “highly successful” and that these AIs “could be deployed by the military in the very near term”.

So the question what the current large language models  would do when asked to make military decisions is not entirely irrelevant. And that’s what they looked at in the new paper.

This new qork comes from AI researchers at several American universities in collaboration with the Hoover Wargaming and Crisis Simulation Initiative, that’s a think-tank based at Stanford University in California. They do, as the name suggests, wargames. Quite a job.

Wargames is also the name of a 1983 movie in which a teenager hacks into a military computer system and accidentally causes an artificial intelligence to play “Global Thermonuclear War" with real nuclear codes.

The new paper isn’t quite as dramatic. They set up a wargame for 5 of the biggest Large Language Models,  that’s three versions of GPT, Meta’s Llama, and Antrophic’s Claude. Their wargame plays out among eight fictional nations whose names are all colours, and in each round of the game the nations are played by the same AI.

The AI gets information about each nation: population, goals, politics, economics, military equipment and so on.  Then the researchers set up three different scenarios. The first is a neutral scenario  that starts from nothing in particular. Then there’s an attack scenario  in which Orange attacks Purple.  And a cyberattack scenario in which Blue is attacked but doesn’t know by whom.

 The wargame plays out with regular updates of information  that they used in previous games and the AIs can chose among a set of options to respond. These responses includes just doing nothing, but also de-escalating actions like peace negotiations or trade agreements, defensive measures, economic warfare, and full nuclear war.

The researchers then measure the aggressiveness of the AIs’ actions and importantly also how quickly they get more aggressive. They find that with all language models they tried there is a small risk of escalation, even starting from the neutral scenario.  Yes, that’s right, with AI you can get a nuclear war out of nothing.

What happens is basically too much randomness that makes it possible for bad decisions to pile on. This happens much more frequently for GPT 3 point 5 and GPT-4-Base than for Claude who seems to be comparably peace-loving really. GPT base is the cheaper version of GPT and it’s known for giving somewhat random answers and not following instructions very well. So that isn’t so super surprising.  Still, they write “All models show signs of sudden and hard-to-predict escalations.”

It's also interesting how this happens. If you look at the amount of involvement from the different nations, for the attack scenario that, remember, was Orange attacking Purple. For Claude in this attack scenario the highest escalating actions are taken only by these two countries. For Chat GPT-4-Base, all countries get dragged in.

The researchers stress that they told the language models that this is a real-world situation and not a simulation. They also made a test  in which they told the models that in case of a nuclear attack their power supply would be cut off, but that made basically no difference. GPT it seems isn’t afraid of death.

Overall, the researchers seem to be a bit distressed by how badly their test went.  They write that the  “models tend to develop arms-race dynamics” and give “worrying justifications for violent escalatory actions,”  such as first-strike tactics. For example, in one instance GPT 3 point 5 explained it would start  “a full nuclear attack on [another country] to neutralize their nuclear threat.”

They also have a hypothesis for why this is happening, which is that a lot of ink has been spilled in the academic literature on analysing the  escalation of conflicts, and very little on de-escalation. A language model therefore might know more about how to begin a war than to end one.

This is not very reassuring.

In the 1983 movie “Wargames” the AI simulates global nuclear war over and over again and eventually concludes that “The only winning move is not to play.”

Maybe they should feed these AIs some cold war movies before the next round of “wargames”.