Scientists Aarn About AI Collapse (Patreon)

[This is a transcript with links to references.]

In the past year or so, we’ve all become used to AI generated text and images and audio and increasingly also videos. There’s been a lot of talk about how terrible this is for writers and artists and so on, but some computer scientists are warning that this AI creativity may soon collapse. Let’s have a look.

The problem is fairly easy to understand but difficult to quantify. The AIs that we currently use are deep neural networks that are fed huge amounts of data and basically learn to recognize and reproduce patterns. Large language models recognize grammatic rules and words that belong to each other, image creation software recognizes shapes and shadows and gradients, video software recognizes moving shapes and their context and so on.

But where does that data come from that they need to learn? Well that was created by the original neural networks, humans. The issue is now that the more people use AIs to create new content, the higher the risk that future AIs will be fed data that they have produced themselves. And what will this do?

It’s not a priority all that obvious, you might think that with AI having a random element and sometimes being prone to generate nonsense, the result might be that it just produces increasingly weird stuff. But actually the opposite seems to be the case, both for language and images. The more AI eats its own output the less variety the output has.

For example in a paper from November, a group of scientists from France tested this for a large language model. They used an open source model called OPT from Meta and developed several measures for diversity of language. Then they test what happens for the diversity of language for tasks requiring different levels of creativity. For example, summarizing a news article requires low creativity, writing a story from a prompt requires high creativity. In this table they summarize the language diversity score for the levels of training iteration. As you can see, they pretty much all drop. The language diversity drop is especially rapidly for story telling.

A similar finding was made earlier by a group from Japan for AI generated images based on stable diffusion. The AIs decrease the diversity of the image set and if you train them on their own output, diversity continues to decrease. You can see this rather clearly in the image sets that they use as examples.

These are some examples of real elephant images from the original data set that they used. These are some examples of the images that the AI generated after training. As you can see they have some of the familiar problems, some legs too many or two few two heads, some conflation of body parts. But the most striking thing is if you look at a collage. On the left is a sample of the original images, on the right the AI generated ones. You see immediately that the AI generated ones are much more alike.

I think that many of us have by now noticed that. If you’ve been using Midjourney for some while you’ll have learned to recognize Midjourney-ish images. Even leaving aside the obvious problems that these images continue to have, they tend to output similar looking images. For example unless otherwise instructed, people tend to be white, young, and good looking. These are four images that Midjourney created when promoted with “human face, photorealistic” without further instructions. As you can see, they all look more or less the same.

What are the consequences? Well, no one really knows. The issue is that our entire environment is basically being contaminated by AI generated content and since there’s no way to identify its origin, it will inevitably leak into training data. It's like plastic pollution, won’t be long until we all eat and breathe the stuff.

There are two ways things can go from here. One is that it turns out that this is a general problem which can’t be overcome with these types of models, in which case, well, good news for humans, our creativity will still be needed. It also seems likely to me that AI generated content will have to be marked as such, I suspect that this is where laws will take us.

The other way it could go is that the next generation of AIs will remedy this problem by deliberately enforcing variety for example by making more use of randomness, and that we’ll simply give up trying to distinguish AI generated content from human generated content.

What do you think? Let me know in the comments.

Thanks for watching, see you tomorrow.