Whatever happened to string theory? (Patreon)

[This is a transcript with links to references.]

String theory. It was supposed to be physicists’ crowning achievement,  a theory that explains no less than everything, with just one simple and elegant idea: it’s all strings. You and I, matter and space, the most fundamental nature of reality, all a big tangle of strings.

It was a beautiful idea, no doubt, and thousands of physicists spent decades on it. But it didn’t quite go according to plan. String theory became extremely controversial about 20 years ago during a phase that’s been dubbed the “String Wars”. Then it kind of fizzled out. What happened? What were the string wars? And what are string theorists doing now? That’s what we’ll talk about today.

The first half of the 20th century was the golden age of physics. One discovery chased another and physicists rapidly developed new theories to explain their observations. Einstein went from special to general relativity, which aced all tests. Quantum mechanics turned to quantum field theory. And eventually, in the early 1970s, physicists completed the standard model of particle physics.

The standard model contains three different forces that physicists currently believe to be fundamental. That they’re fundamental means that they don’t arise from anything else, it’s rather that the rest arises from them. First there is the electromagnetic force, that is itself a combination of the electric and magnetic force. And then there are the strong and weak nuclear forces. But the standard model doesn’t contain gravity. That’s still described by Einstein’s General Relativity.

Most physicists thought then, and still think today, that the standard model can’t be it. A fully satisfactory theory of nature would have to describe all the fundamental forces, the three in the standard model plus gravity, as one. Such a theory is known as a “theory of everything”.

After the completion of the standard model, string theory swiftly became the hottest contender for this theory of everything.

It was an accidental candidacy. The idea of string theory originally came from nuclear physicists who wanted to describe what happened inside protons and neutrons, the constituents of atomic nuclei, and similar composite particles. Protons and neutrons are each made of three quarks, held together by gluons. But the gluons don’t just form a homogeneous soup, they preferentially flow between certain routes between the quarks. These routes are known as “gluon flux tubes”. They’re a little like strings. And so string theory was invented.

This version of string theory still exists today as a bottom-up approach to understand what gluons do, but the string theory that we are concerned with is a different one. You see, when physicists studied what those strings do, they found that some of them behave like a graviton, that’s a quantum of the gravitational interaction.

It’s a particle which has never been observed, not then and not now, but that could be the start of giving quantum properties to gravity. At the same time, strings were so versatile that they could also behave like particles that make up matter and those who make up light. Strings could do all of it. It was what they’d been looking for – a theory of everything.

But as soon as string theorists got started, problems began to pop up. First, there was the issue that the vacuum of their theory wasn’t stable. The equations said that it was falling apart into pure energy, destroying the universe along with it. Now, I don’t know about you, but for all I can tell this hasn’t happened. To remedy this problem, string theorists needed a new symmetry, “supersymmetry.”

Supersymmetry fixed the problem just fine, but it required that each of the particles in the standard model had a partner particle which couldn’t be one of the already observed ones. So now the problem was where all those supersymmetric partner particles were. String theorists claimed that they full well exist, we just haven’t been able to see them because they’re too heavy.

You see, producing a heavy particle requires a lot of energy, and for that you need big particle collider. String theorists simply said we haven’t built a collider large enough to see those supersymmetric partner particles.

Plausible enough you might say, but even so, supersymmetry predicted something else, a type of process that can’t happen in the standard model. It involves what’s known flavour-changing neutral currents. Evidence for those processes should have shown up in the early 1990s at the Large Electron Positron collider at CERN. It did not. And so, string theorists added another fix to their theory, R-symmetry, to make their equations agree with observations again.  

Then there was the problem that string theory required a total of 10 dimensions of space to properly work. Unfortunately, it seems that the space we find ourselves in has merely 3 dimensions. String theorists explained away the extra dimensions of space by saying they are rolled up to sizes so small that we can’t see them. Again this works because measuring something small requires a lot of energy, and we might not have noticed these small dimensions because we haven’t been able to achieve particle collisions with sufficiently high energy.

Now when I say they rolled up those extra dimensions that might suggest each of them is like a straw and there’s only one way to do it. But actually there are many different ways to do it. Already in two dimensions you have a torus or a sphere as example. And if you have 6 dimensions there’s a huge number of ways to roll these dimensions up, some hundred thousand or so. At this point, the idea that string theory was unique and would just spit out the standard model went out of the window.

The problems did not stop there. String theory works best in a universe with a negative cosmological constant and string theorists originally just assumed that’d be so. The cosmological constant, if you remember, is a constant of nature that determines how the expansion of the universe changes, whether it gets faster or slower. The expansion of our universe gets faster, and that means the cosmological constant is positive, exactly the opposite of what string theory requires.

String theorists found a way to fix that problem, but that introduced even more ambiguity. It also turned out that string theory does not actually reproduce Einstein’s theory of general relativity, but gives rise to a modified version of it, which again runs into conflict with experiment, unless one adds further requirements.

If all of this sounds rather unappealing, it’s because it is. But at this point there were already thousands of string theorists at work, and they didn’t find it unappealing at all. Because string theory had one thing going for it that I think most people don’t appreciate enough. It’s that string theory is mathematically an extremely rich theory. There was a lot to discover. It was completely unexplored territory, there was just so much to learn and understand and, of course, to write papers about. I know this might sound a little strange, but I think this is why the string theory community grew to such size, it’s simply because it was just mathematically so rich.

But all this paper writing didn’t help string theorists find that theory of everything they’d been looking for. There were just too many versions of string theory now, an estimated 10 to the 500. And since they couldn’t find one that actually reproduced the standard model and general relativity, they postulated all of them exist. This is the so-called string theory landscape.

I’ve always found that to be a particularly idiotic move. Just because you can’t figure out which theory describes reality doesn’t mean all of them are real. And in any case, it didn’t solve any problem because they did, as a matter of fact, not have the theory they were looking for.

In the late 1990s then, two things happened almost simultaneously that split the history of string theory into two branches. The one thing that happened was a desperate attempt to squeeze testable predictions out of string theory which backfired, eventually leading to the string wars. The one was Juan Maldacena’s discovery that some simplified cases of string theory are similar to already known theories which describe certain types of matter. This is the so-called AdS/CFT correspondence. I’ll tell you about the string wars first and if you’re still awake afterwards, I’ll tell you about AdS/CFT.

As we saw, string theory had been repeatedly amended to avoid conflict with experiment. Many physicists were highly critical of this procedure. Richard Feynman for example didn’t hold back with his opinion about string theory: “I think all this superstring stuff is crazy and is in the wrong direction… I don’t like that for anything that disagrees with an experiment, they cook up an explanation—a fix-up to say, “Well, it might be true.””

But then, in the late 1990s, a solution to string theorist’s problems appeared from the ranks of particle physicists. Some of them, notably Nima Arkani-Hamed and Lisa Randall, argued that actually the dimensions of string theory might be so large that the Large Hadron Collider should be able to test them.

At this time, the Large Hadron Collider, LHC for short, was the planned next supercollider at CERN. It was still more than a decade away from starting operations. There wasn’t any reason why the dimensions of string theory should be so large as to show up there, so it was not an assumption they should have made.

But the vast majority of physicists have no training in the philosophy of science, and so the idea that string theory could be tested soon became very popular very quickly.

Within just a year or two, there were literally thousands of particle physicists and astrophysicists who claimed they were using string theory to make predictions for upcoming experiments. The key for getting this nonsense past peer review was to invent reasons for why this supposed evidence for string theory had not been seen so far, but would show up in the next experiment. It’s always the next experiment that will find it, and in the 1990s, that next experiment was the Large Hadron Collider.

Putting “string theory” in a grant proposal became a way to get your research funded. I myself wrote my PhD thesis on how to test those large extra dimensions of string theory at the LHC. Then I unceremoniously dropped off the bandwagon.

And would you believe it, neither the LHC nor any other experiment saw even a hint of these supposedly large extra dimensions. Still, a lot of people made career with it. Lisa Randall became one of the best-cited physicists ever for her work on it, and Nima Arkani-Hamed won one of the breakthrough prizes for it. It wasn’t because of the scientific relevance -- there wasn’t any. It was because their papers on the idea had been cited ten thousands of times by other physicists, and some people were very impressed by that. It was like they thought if so many physicists talk about it, there must be something to it. But there wasn’t anything.

I wasn’t the only one back then who had the feeling that something was going badly wrong in the foundations of physics. In 2006, two books appeared almost simultaneously, taking issue with string theory. One was Lee Smolin’s “The trouble with physics”, the other Peter Woit’s “Not Even Wrong”.

Smolin used the opportunity to push his own favourite approach to quantum gravity, loop quantum gravity, though it was apparent already that this wasn’t going anywhere either. Woit focused on some technical problems that string theorists of course claimed were solvable, though they never solved them.

The fallout was rather ugly. Multiple prominent string theorists used ad hominem arguments against Smolin and Woit in attempts to discredit the two, rather than addressing their arguments because there really wasn’t any way to address them. There were few string theorists involved in this exchange and it’d be unfair to judge all of them by what a few did, but life isn’t fair. The unprofessional behaviour of people like Lenny Susskind and Michael Duff in response to very valid criticism from Smolin and Woit significantly contributed to the demise of string theory.

This episode became informally known as the “String Wars” and though I’m not the kind of person who likes picking on language, I find this term unfortunate because it seems to belittle the suffering of people in actual wars.

Not much came out of this kerfuffle at the time because many physicists were quite convinced that the LHC would find evidence of supersymmetry, giving a much-needed boost to string theory. But in 2010 the LHC turned on, didn’t find any evidence for supersymmetry or extra dimensions or string balls or gravitons or what have you, and the bubble in which string theory had been testable burst. At this point I was glad I wasn’t working on this stuff anymore, though I’d thrown away my chance of getting tenured as several friends never cease to remind me.

The result of the string wars and the following failure of the LHC to see even a shred of evidence in support of it, was that the branch of string theory which contained the last remains of the original idea, to find a theory of everything, basically died off.

Let’s then talk about the other branch, the one with the difficult name AdS/CFT. Starting with: what does that even mean?

Remember that our universe has a positive cosmological constant. The mathematics which describes such a universe is known as a De-Sitter space. It’s named after Willem De Sitter, a Dutch mathematician and physicist whose beard’d make even Rohin jealous. A universe with a negative cosmological constant, then, is called an Anti-de Sitter space, AdS for short. It not that it’s a space which petitions against De-Sitter, it just means the sign of that constant negative.

String theory works well in these Anti-De Sitter spaces because there are some additional mathematical techniques that can be used. In 1997 now, Juan Maldacena pointed out that some approximations to string theory in these Anti-De Sitter spaces look mathematically similar to a more familiar theory, a type of Conformal Field Theory, CFT for short, in a space with one dimension less. And, importantly, this other theory, the Conformal Field Theory, doesn’t contain gravity anymore. His idea became known as the AdS/CFT correspondence.

I know this is all very technical, but just to give you a simple example. If you have quantum gravity in a universe with a negative cosmological constant, this Anti-De Sitter space, with 3 dimensions of space plus one dimension of time, then that would correspond to one of those conformal theories without gravity in 2 dimensions of space plus one dimension of time.

It’s sometimes said to be a sort of “holographic” principle because it expresses this 3-dimensional space with information on a 2-dimensional slice, kinda like holograms, if you don’t think too much about it. Which of course I’ve done, and therefore I have a video explaining why it’s actually not like a holograph but I digress.

The AdS/CFT correspondence is a big deal because these conformal field theories are similar to some theories that physicists use to describe matter, but matter without gravity. This means it removed the problem of having to quantize gravity because physicists could now say, it’s mathematically the same as a theory that doesn’t have gravity! At least if you’re interested in quantum gravity in a universe we don’t inhabit.

But Maldacena was really saying two things. One is that we might be able to use some familiar mathematics from those matter theories without gravity to describe quantum gravity in this Anti De Sitter space. And second, that we might use string theory to say something about the behaviour of matter without gravity, or when gravity has a very small influence so that it can be ignored.

This second option is interesting because that gravity can be ignored is the case in almost all experiments in particle physics and nuclear physics. It’s not that gravity isn’t there, but it’s so weak compared to the nuclear and electromagnetic forces that one doesn’t need to calculate what it does.   

To keep track of where string theorists were going with this, note that neither of those two options has anything to do with the original goal of finding a theory of everything for our universe. We don’t live in an Anti De Sitter space, and using string theory to describe matter without gravity in our universe helps us nothing with unifying the interactions, exactly because that formulation doesn’t contain gravity.

For some while after Maldacena put forward his idea, string theorists would insist that actually Anti-De Sitter space is kind of similar to De Sitter space, so we shouldn’t worry too much about the sign of the cosmological constant.

I have always found this argument funny because it’s obviously wrong. There are many solutions in Anti-De Sitter space which just don’t exist for De Sitter space. The most famous example is what’s called a planar black hole. That’s a black hole for which the horizon is an infinitely extended plane.

Yes, it’s an infinitely large black hole. Such a thing can’t exist in our own universe, and personally I think it’s for the better because I’d rather not accidentally walk into a black hole on the way to the supermarket. But in Anti-De Sitter space it can exist and indeed it’s used all the time in calculations. So clearly they do need the cosmological constant to be negative. If they didn’t need it to be negative, why would they do it.

Be that as it may, string theorists made a few attempts at using the correspondence for De Sitter space, so for a universe like our own. It wasn’t very convincing, and I don’t think it ever will be, but then I’m just some random YouTuber, so what do I know.

And so, the only thing that remained of string theory and that has survived until now is the AdS/CFT branch. There are some people left who still work on that original dream of a theory of everything but they’re few and not getting anywhere.

What are they doing with this AdS/CFT stuff? Well at first they said it’d be useful to describe the quark gluon plasma. That’s a hot soup of quarks and gluons that probably existed in the early universe. Small amounts of it can be produced if one collides two large atomic nuclei, usually gold or lead, at high energy. They sometimes do this at the LHC. String theorists made some predictions for that, they turned out to not fit the data, and then they quietly buried the attempt.

They have since claimed that ads cft is good to describe some types of materials that include strange metals some of which are high temperature superconductors. This makes for a sexy motivation because we don’t really understand what makes some materials high temperature superconductors, and finding out would be a big breakthrough with technological relevance. But there isn’t much that’s come out of it either. I don’t want to say nothing’s come out of it, because AdS/CFT methods have indeed proved to be useful for some calculations, it’s just that, you know, didn’t exactly change the world did it?

A more recent development that started a few years ago is that some of the AdS/CFT people want to benefit from all the money that’s going into quantum technologies. So they say that certain configurations of entangled qubits on quantum computers can be approximated by the CFT side of the duality, and then they correspond to a wormhole in that AdS space. This is where the story with the wormhole on a quantum computer came from. Of course there wasn’t really a wormhole in the computer. It’s just that by this mathematical correspondence, you can interpret it as if it was a wormhole in some anti-de sitter universe we don’t inhabit.

For good measure, you can also throw some artificial intelligence at AdS/CFT and this is becoming increasingly popular too. And that’s what most string theorists are doing today: using some mathematical techniques that have come out of string theory and applying them to other things. Basically, making lemon juice after life gave them lemons.

In summary. String theory had a really good motivation, and it was pursued as a theory of everything for good reasons. However, when that didn’t work out, string theorists were slow to get the message and a lot of time and effort was wasted on it. It’s not that string theory turned out to be completely useless. Some techniques have survived and are being used today in related areas of physics.