It’s not an exaggeration to say that the flood of data in the 20th century for theoretical physics has now effectively vanished, and that leaves theoretical physics in a limbo as they figure out which TOE is true.
String theory, Loop Quantum Gravity and more theories try to duke it out, but almost no data can discern between these theories.
So I’m asking you, LessWrongers to figure out how to weight credences between these theories to at least identify the most probable theory for our universe.
(One hint: Supersymmetry, at least in it’s non-broken forms, is unlikely to exist. If Supersymmetry is correct, then it has to be broken in a way that doesn’t solve the Hierarchy Problem.)
I think there’s a rationalist antipattern, where an area is very data-poor, and one then decides that one can come up with the correct theory for that area using reason, e.g. Occam’s razor/Bayesian updates/minimum description length/etc..
My experience is that this works surprisingly poorly in practice. Probably the reason is that people massively underestimate just how many models can be made which fit through the data one has observed.
In particular, one thing that has an especially high tendency to go badly is this:
To illustrate how that can go bad, consider a biased coin with 60% chance of ending up tails, and imagine you flipped it 10 times. What’s the most likely sequence you could observe from this? HHHHHHHHHH. But is this a typical sequence, whose properties you can use to probably predict things right? No, you’d often (not always, depends on various obvious things) be better off with something like HTHHHTHTTH as your prototypical sequence, even though it is less likely, because its aggregate properties are more accurate.
Though you’d be even better off by integrating over all the possible sequences weighted by the probability, which in this metaphor corresponds to keeping an open mind to different possibilities while also being very skeptical about any 1 of them.
This is a good point, but also kind of an oversimplification of the situation in physics. Imagine Alice is trying to fit some (x, y) data points on a chart. She doesn’t know much about any kinds of function other than linear functions, but she can still fit half of the points at a time pretty well. Half of the points have a large x coordinate, and can be fit well by a line of positive slope. Alice calls this line “The Theory of General Relativity”. Half of the points have a small x coordinate, and can be fit well by a line of negative slope. Alice calls this line “Quantum Field Theory”. Collecting data in the region in between is very difficult for technical reasons, and so Alice doesn’t have any data there yet. But it looks like if she extended the lines until they met, they would end up making a kind of “V” shape.
This is a huge problem for her, because there is no linear function that gives a good fit on both sets of data points. Whatever the true function, it must somehow be “non-linear”, whatever that means. In order to go through both sets of points, the function must somehow “bend”. Alice can kind of imagine what a bendy function ought to look like, but when it comes time to put it into mathematics, she’s suddenly stuck. All the functions she’s ever seen can be written in the form y=mx+b, but how could there be a value for m that works for this kind of function? No real number works, and she tried complex numbers, but none of those worked either. Now Alice is trying to think of other number systems extending the reals that might contain a good value of m.
Then Bob comes along, and says “look, you’re attempting an impossible problem. There are way too many functions that go through all the data points we have so far. In order to distinguish between them, we have no choice but to try and collect some data in middle region, in between the two clusters of data points that we have so far. We have an oversupply of theories, and what we really need is data.”
To this Alice replies, “yes, in some sense it’s true that the data we currently have is insufficient. But I also wouldn’t say that we have an oversupply of theories. We know that the true function must fit well to all of the data points we already have, but we can’t actually explicitly write down even one single function like that. We have various hand-wavey notions of how one might eventually be able to write down such a function and calculate its values for various x coordinates, but none of those notions have actually delivered. Our understanding of the problem just isn’t good enough yet. Relative to our current mathematical capabilities, the problem isn’t under-constrained, it’s so over-constrained that there are 0 solutions, and no one is even close to finding one. More experiments are always good, but I’ll only agree that we can’t make progress without them once someone can find me two distinct candidate theories that are both internally consistent, and that fit all the data we’ve already taken.”
That is a fair point.
Near the end of his life, Feynman said, we do have plenty of data, it’s all the unexplained parameters in the standard model.
Another hint: Witten once said, there has never been a theory which contained as many of the ingredients of reality as string theory (fermions, gauge fields, gravity), that turned out to be unrelated to reality.
Loop quantum gravity, on the other hand, is extremely overrated. The nonperturbative form of the theory has been incapable of recovering space-time.
Penrose’s twistors are great, they have revealed deep new structures in field theory and string theory, and even have some relationship to the Ashtekar variables that are employed in loop quantum gravity (it’s just LQG’s method of quantization which is probably wrong).
The old paradigms of supersymmetry appear to be wrong. The old paradigms of dark matter are probably wrong too, given the number of correct predictions that come from “MOND”. However, there are many known variations on these ideas, and we are also simply very far from knowing all possible forms of field theory and string theory.
The odds are that we are living in some kind of string theory possible world, that is slightly or markedly different from the ones that have received most of the attention in particle physics. However, non-string-theory hypotheses deserve to have people investigating them too, they may turn up overlooked pieces of the puzzle.
To the extent that these opinions of mine provide appropriate guidance, I can’t say it’s due to the use of any rationalist heuristics. It just comes from lots of study and thinking about the subject, and having Internet guides, for the best orthodox wisdom (Lubos Motl) and for promising ideas on the fringes (Marni Sheppeard, RIP).
I think you are asking the wrong question. What do you want to use these credences for? Are you planning to work on fundamental physics? Then the relevant question is something like “In what area am I likely to make progress?” or “Which of these fields offers the best opportunities for advancement?” If one does not expect to ever know what model, if any, will successfully unite QM and GR, then asking for credences is a fun but pointless speculation.
(Now, to express my personal speculative view for fun, I suspect that none of the existing models come close to anything like a “Theory of Everything”, we are short a couple of paradigm shifts and about 10 orders of magnitude off in experiments, but there will be progress once experiment catches up by e.g. measuring gravcats.)
Now my personal credences are 55% String Theory with broken Supersymmetry, about 1-5% to Loop Quantum Gravity, and about 40-44% credence in other theories.
That means String Theory still is the most likely theory that applies to reality, but definitely not overwhelmingly confidently likely.
One other problem for String Theory is the black hole information paradox has been solved solely by normal physics, that is there is no need to posit strings to solve the paradox. Conjunctions are never likelier than their conjuncts, so solely solving the black hole information paradox using normal physics is strictly more likely than positing String Theory and the AdS/CFT correspondence.
Here’s a link:
https://www.quantamagazine.org/the-most-famous-paradox-in-physics-nears-its-end-20201029/
Well: this “solution” to the paradox was discovered by working within string theory. Then they found a way to carry out the argument at a level of approximation that doesn’t need to know what the fundamental theory is. That is the only sense in which they were able to dispense with string theory. They don’t actually have an alternative fundamental theory in which the argument can also be carried out.
Meanwhile, I put “solution” in quotes because it is far from clear what’s going on here, and how it relates to other “solutions” that have been proposed, like Mathur’s fuzzballs, Hawking et al’s soft quantum hair, and Papadodimas and Raju’s state-dependent observables. For example, is the island of quantum information in the black hole interior, a baby universe that becomes genuinely disconnected from the original space-time, or is it just a brane that buds off while remaining immersed in a common higher-dimensional space? This is related to a larger debate within string theory, about how to interpret the lower-dimensional models that are being used here.
To explain to everyone, the reason the black hole information paradox has been solved is that 2 things happened:
First, the Page Curve was resolved as, yes black holes do preserve and shed information, thus we’ve resolved whether information is destroyed as Hawking and Preskill thought, or does unitarity prevail in black holes, preserving information. Now we know that information must be preserved. Incredibly they even got it to work in our own flat universe.
Second, they got rid of both string theory and the AdS-CFT Correspondence/Holographic universe assumption, removing 2 burdensome details from the solution. It’s inspired by string theory and the AdS/CFT Correspondence/Holographic universe theories, but they don’t rely on these assumptions to solve the paradox. We’ve finally managed to get the answer without relying on burdensome details.
Now how it happens is unfortunately unsolved, and here the best theories like wormholes are pretty speculative. But we don’t need the how in order to appreciate the answer to the original Black Hole Information Paradox.
I have just done some further research, and found a rather serious criticism of the idea that “quantum extremal surfaces reproduce the Page curve, and that explains everything”. The claim is that entanglement islands only show up in theories with a massive graviton—which is very unlike our world. This claim is being emphasized by the string theorist Suvrat Raju, most recently in section 4.2 of this paper from last year. He actually mentions the article from Quanta Magazine, as an example of a “misleading” “popular description”.
His own position is that everything about real-world quantum black holes is explained by an extra nonlocality of information in quantum gravity (what he calls “holography of information”); that the Page curve is a property of any ordinary quantum system in which entanglement is leaking across a boundary; and it’s because massive gravity doesn’t possess the property of holography of information, that the Page curve shows up in that case.