I think the problem is that you ignore the idea that science works via paradigms. Even if there’s a possible paradigm besides string theory that would produce more progress, there are a lot of different things that people who aren’t working on string theory could work on. Most of them won’t lead anywhere.
If a new paradigm could be found that has more potential, that paradigm would have new low hanging fruit.
However, researchers that would write papers about that low hanging fruit, might have trouble getting published in journals of the old paradigm because they are solving problems of interest to the new paradigm and not problems of interest of the old paradigm. Getting funding to work on problems of a new paradigm is also harder.
OK I will try to take that idea where as far as it can go. Lets assume that a lot of progress is stalled by the difficulty in overturning paradigms. For that to be the complete reason, the difficulty in overturning paradigms would have to not increase as the knowledge and maturity of the field increased. Otherwise that difficulty would still be side effect of the level of advancement in the field and just another argument in favor of diminishing returns as a field advanced.
Some fields its easy to overturn paradigms if there a very simple public metric like high jump with the Fosbury flop—it was immediately clear it was superior. If a field doesn’t have such a metric then its harder. Also if there is existing results that must be achieved by a new paradigm before its even clear that it is superior. An example of this is potentially AI where a new architecture may be superior on small data sets, but not so on large sets and if this happens a lot, then its hard to know you have an improvement without actually running costly tests. Additionally if the politics of the field is setup to make it hard for new paradigms to flourish then that is also a source of difficulty.
So for physics? Its clear there are a lot of existing results that need to be reproduced for a theory of quantum gravity or similar to be taken seriously, which both makes it difficult to get traction, but also gives external credibility. For example, today a theory has to be compatible with General Relativity and give Newtonian physics at low speed/energy etc. In Einstein’s day, the barrier would seem to have been lower.
So is there a way for a rational community to overcome these issues? As people note there are many ideas floating around for new paradigms but no easy way to find the promising ideas. That seems more like a structural problem than something politics could fix. The community would perhaps have to come up with a list of qualities or successful calc that a new paradigm could be judged on short of trying to reproduce all of physics? Anyway its not clear to me that there is large progress that could be made in that area.
So yes even if the slowdown is because we don’t have the best paradigm there needs to be an efficient way to find such paradigms that doesn’t get more inefficient as the field advances to count against the general diminishing returns argument.
In many cases, new paradigms care about different metrics than older paradigms. In the beginning, successful new paradigms usually don’t fulfill the qualities that heterodox researchers in the field are looking to. You might want to read Thomas Khun’s “The Structure of Scientific Revolutions”.
There are ways you can make it easier to overturn paradigms. You can change the way research is funded. You could change the grant-making processes in a way that makes it easier for very smart young people to pursue research agendas that heterodox old people don’t find interesting.
There’s the Max Planck quote of “Science advances one funeral at a time”. In the last decades, old researchers got more power over which research gets conducted than back in 1950 when Planck wrote his autobiography.
Sure but does this actually apply to physics? Can anyone suggest different metrics, or is there broad agreement with everyone what the major physics problems are, because it seems like there is. E.g. the non conventional people don’t say dark energy isn’t important, they have different explanations for what it is. Everyone agrees the nature and origin of time/entropy etc is important.
Can you give examples of very smart young physicists complaining they are pushed into old ways of thinking? Are you prepared to give an justify a % difference that such things would make?
For example the comment by “Mitchell_Porter”
The idea that progress is stalled because everyone is hypnotized by string theory, I think is simply false, and I say that despite having studied alternative theories of physics, much much more than the typical person who knows some string theory.
No-one has pushed back against that here. I see your position as a theory that we need to gather evidence for and against and decide on a field by field basis. In this field I only see data against that position.
There are people like Sabine Hossenfelder who think that there are experiments we could run that we currently don’t run, because they don’t fit into the string physics paradigm.
If you look back through history, in most times you would not see data that shows that a specific non-hegemonic scientific paradigm is better than the hegemonic paradigm. We usually only have clear data that a different paradigm is better when the old paradigm stops being hegemonic.
Ok can we put some rationality to this. Your prior seems to be that when a field is stuck, it is almost entirely because of politics, hegemon etc. So how do we update that prior?
What would you expect to see in a field that is not stuck bc of inertia etc. You would still get people like Hossenfelder saying we are doing things wrong, and such people would get followers.
You suggested metrics before but havn’t provided any that I can see.
Evidence I take into account:
#1 There is not a significant group of young capable researchers saying we are taking things in the wrong direction, but a smaller number of older ones. Unless you are going to go so far as to say they are afraid to speak out, even anon, then that to me is evidence against your position. There are two capable experts on this blog from what I can see, one enthusiastic about string theory, and another who has investigated other theories in detail. Both disagree with your claim.
#2. There is not broad disagreement about what the major issues in physics are, so unlikely to be disagreements on metrics. You mentioned this as mattering, and if it does, I count this as evidence against your position.
Can you point to evidence that actually supports your prior? I can only see that which opposes it or is neutral. (In all fields no matter how things are progressing you get some people who think the establishment is doing it all wrong and have their own ideas. That can’t count as evidence in favor for a specific field unless it is happening to a greater degree)
I don’t disagree with your position in general. In fact it was clear to me that AI before neural networks was stuck with GOFAI and I believed that NN were clearly the way to go. I followed Hinton et all before they became famous. I saw the paradigm thing play out in front of my eyes there. Physics seems different.
You made statements that you know why the field is stuck. That both contains an assertion of the field being stuck and you knowing the explanation is due to diminishing returns. I made no claim that I know the answer to either of those questions.
#1 There is not a significant group of young capable researchers saying we are taking things in the wrong direction, but a smaller number of older ones.
If you think that theoretical physics is going in the wrong direction and are unlikely going to get to research in a way you think will make progress, there are good reasons for not being not making your PHD in theoretical physics. The strongest disbelievers filter themselves out.
But even among the people who are actually in the field, I don’t see a good reason why you would publically see signs.
You might ask PHD students: “If you wouldn’t need to seek grants and would get a lab with 5 million dollar per year, would you pursue the same research agendas as you are currently do or would you pursue research for which you wouldn’t get grants in the current academic environment?”
It might be interesting to ask that question for researchers in every field and see the responses, but unfortunately I don’t know of any source that asks such a question in a good way.
I followed Hinton et all before they became famous. I saw the paradigm thing play out in front of my eyes there.
It’s certainly possible for their to be fields where it can be obvious to outsiders that a particular paradigm changing approach will work, but that’s not necessary for there to be superior paradigms that could be persued.
I think the problem is that you ignore the idea that science works via paradigms. Even if there’s a possible paradigm besides string theory that would produce more progress, there are a lot of different things that people who aren’t working on string theory could work on. Most of them won’t lead anywhere.
If a new paradigm could be found that has more potential, that paradigm would have new low hanging fruit.
However, researchers that would write papers about that low hanging fruit, might have trouble getting published in journals of the old paradigm because they are solving problems of interest to the new paradigm and not problems of interest of the old paradigm. Getting funding to work on problems of a new paradigm is also harder.
OK I will try to take that idea where as far as it can go. Lets assume that a lot of progress is stalled by the difficulty in overturning paradigms. For that to be the complete reason, the difficulty in overturning paradigms would have to not increase as the knowledge and maturity of the field increased. Otherwise that difficulty would still be side effect of the level of advancement in the field and just another argument in favor of diminishing returns as a field advanced.
Some fields its easy to overturn paradigms if there a very simple public metric like high jump with the Fosbury flop—it was immediately clear it was superior. If a field doesn’t have such a metric then its harder. Also if there is existing results that must be achieved by a new paradigm before its even clear that it is superior. An example of this is potentially AI where a new architecture may be superior on small data sets, but not so on large sets and if this happens a lot, then its hard to know you have an improvement without actually running costly tests. Additionally if the politics of the field is setup to make it hard for new paradigms to flourish then that is also a source of difficulty.
So for physics? Its clear there are a lot of existing results that need to be reproduced for a theory of quantum gravity or similar to be taken seriously, which both makes it difficult to get traction, but also gives external credibility. For example, today a theory has to be compatible with General Relativity and give Newtonian physics at low speed/energy etc. In Einstein’s day, the barrier would seem to have been lower.
So is there a way for a rational community to overcome these issues? As people note there are many ideas floating around for new paradigms but no easy way to find the promising ideas. That seems more like a structural problem than something politics could fix. The community would perhaps have to come up with a list of qualities or successful calc that a new paradigm could be judged on short of trying to reproduce all of physics? Anyway its not clear to me that there is large progress that could be made in that area.
So yes even if the slowdown is because we don’t have the best paradigm there needs to be an efficient way to find such paradigms that doesn’t get more inefficient as the field advances to count against the general diminishing returns argument.
In many cases, new paradigms care about different metrics than older paradigms. In the beginning, successful new paradigms usually don’t fulfill the qualities that heterodox researchers in the field are looking to. You might want to read Thomas Khun’s “The Structure of Scientific Revolutions”.
There are ways you can make it easier to overturn paradigms. You can change the way research is funded. You could change the grant-making processes in a way that makes it easier for very smart young people to pursue research agendas that heterodox old people don’t find interesting.
There’s the Max Planck quote of “Science advances one funeral at a time”. In the last decades, old researchers got more power over which research gets conducted than back in 1950 when Planck wrote his autobiography.
Sure but does this actually apply to physics? Can anyone suggest different metrics, or is there broad agreement with everyone what the major physics problems are, because it seems like there is. E.g. the non conventional people don’t say dark energy isn’t important, they have different explanations for what it is. Everyone agrees the nature and origin of time/entropy etc is important.
Can you give examples of very smart young physicists complaining they are pushed into old ways of thinking? Are you prepared to give an justify a % difference that such things would make?
For example the comment by “Mitchell_Porter”
No-one has pushed back against that here. I see your position as a theory that we need to gather evidence for and against and decide on a field by field basis. In this field I only see data against that position.
There are people like Sabine Hossenfelder who think that there are experiments we could run that we currently don’t run, because they don’t fit into the string physics paradigm.
If you look back through history, in most times you would not see data that shows that a specific non-hegemonic scientific paradigm is better than the hegemonic paradigm. We usually only have clear data that a different paradigm is better when the old paradigm stops being hegemonic.
Ok can we put some rationality to this. Your prior seems to be that when a field is stuck, it is almost entirely because of politics, hegemon etc. So how do we update that prior?
What would you expect to see in a field that is not stuck bc of inertia etc. You would still get people like Hossenfelder saying we are doing things wrong, and such people would get followers.
You suggested metrics before but havn’t provided any that I can see.
Evidence I take into account:
#1 There is not a significant group of young capable researchers saying we are taking things in the wrong direction, but a smaller number of older ones. Unless you are going to go so far as to say they are afraid to speak out, even anon, then that to me is evidence against your position. There are two capable experts on this blog from what I can see, one enthusiastic about string theory, and another who has investigated other theories in detail. Both disagree with your claim.
#2. There is not broad disagreement about what the major issues in physics are, so unlikely to be disagreements on metrics. You mentioned this as mattering, and if it does, I count this as evidence against your position.
Can you point to evidence that actually supports your prior? I can only see that which opposes it or is neutral. (In all fields no matter how things are progressing you get some people who think the establishment is doing it all wrong and have their own ideas. That can’t count as evidence in favor for a specific field unless it is happening to a greater degree)
I don’t disagree with your position in general. In fact it was clear to me that AI before neural networks was stuck with GOFAI and I believed that NN were clearly the way to go. I followed Hinton et all before they became famous. I saw the paradigm thing play out in front of my eyes there. Physics seems different.
You made statements that you know why the field is stuck. That both contains an assertion of the field being stuck and you knowing the explanation is due to diminishing returns. I made no claim that I know the answer to either of those questions.
If you think that theoretical physics is going in the wrong direction and are unlikely going to get to research in a way you think will make progress, there are good reasons for not being not making your PHD in theoretical physics. The strongest disbelievers filter themselves out.
But even among the people who are actually in the field, I don’t see a good reason why you would publically see signs.
You might ask PHD students: “If you wouldn’t need to seek grants and would get a lab with 5 million dollar per year, would you pursue the same research agendas as you are currently do or would you pursue research for which you wouldn’t get grants in the current academic environment?”
It might be interesting to ask that question for researchers in every field and see the responses, but unfortunately I don’t know of any source that asks such a question in a good way.
It’s certainly possible for their to be fields where it can be obvious to outsiders that a particular paradigm changing approach will work, but that’s not necessary for there to be superior paradigms that could be persued.