I can’t think of a reason to privilege the hypothesis that they were distorted by the trauma mechanism in particular.
Okay, at the risk of starting a big side argument that’s not cruxy to the OP, I’m going to push for something controversial here:
I think it’s a mistake to view privileging the hypothesis as a fallacy.
Eliezer gives a fictional example like so:
“Well… we have no idea who [committed this murder]… no particular evidence singling out any of the million people in this city… but let’s consider the hypothesis that this murder was committed by Mortimer Q. Snodgrass, who lives at 128 Ordinary Ln. It could have been him, after all.”
Yes, nakedly without any context whatsoever, this is dumb. Why consider Mr. Snodgrass?
But with some context, there absolutely is a reason:
There’s some cause to why the detective suggested Mr. Snodgrass in particular.
It’s not just that there might be some cause. There must be.
Now, there are lots of causes we might not want to allow:
Maybe he had a random number generator pick out a random person and it happend to be Mr. Snodgrass.
Maybe he personally hates Mr. Snodgrass and sees an opportunity to skewer him.
Maybe he knows who did commit the murder and is trying to cover for the real murderer, and can tell that maybe Mr. Snodgrass could be framed for it.
But with the exception of the random number generator, there’s necessarily information entangled with the choice of poor Mortimer.
Any choice of a hypothesis necessarily must be kind of arbitrary. John Vervaeke talks about this: he calls it “relevance realization”. It’s a pragmatically (if not theoretically) incomputable problem: how do you decide what parts of reality are relevant? As far as I know, all living systems come up with a partial functional answer based on just grabbing some starting point and then improvising, plus adding a bit of random variance so as to catch potentially relevant things that would otherwise be filtered out.
So if you add a caveat of “But wait, this is privileging the hypothesis!”, you’re functionally putting sand in the gears of exploration. I think it’s an anti-pattern.
An adjacent and much more helpful process is “Huh, what raised this hypothesis to consideration?” Quite often the answer is “I don’t know.” But having that in mind is helpful! It lets you correct for ways in which the hypothesis can take on a different weight than it should. E.g., it helps a lot to know the detective used a random number generator to pick out Mr. Snodgrass.
(That particular case also makes the guess useless because that random number contains no information — whereas the detective’s personal hatred of Mr. Snodgrass does contain information!)
Now, you mention that you “can’t think of a reason” to privilege the trauma hypothesis. And yet I gave a post full of reasons. Is it that you needed some reason before what I said to consider it? If so, I think that’s the “privileging the hypothesis fallacy” fallacy. With what methods would you pick out a hypothesis worth considering? Why are you privileging those methods; what justifies them as hypothesis-pickers? What justifies those answers for your meta-hypothesis pickers? Etc. It’s not a coherent starting point for a functional epistemology that a human can actually do AFAICT.
The problem that privileging the hypothesis is pointing out is when someone fixates on a possibility and then uses that fixation to filter out or significantly bias evidence. But that’s not what’s happening here. I’m saying “Hey, I keep seeing a pattern that sure looks like it could be X, and if so then there’s a real and relevant problem being neglected. Here’s a suggestion about a way to check whether it’s X. Does that check work?” That’s absolutely not the kind of fixation that Eliezer was warning about!
There is, however, a different question that strikes me as relevant (!): “What caused you, Val, to point at this trauma model in particular? Why are you looking at this possibility in particular? Why this X?”
I don’t want to try to add an answer to that question in this comment. I think that’ll create noise as to what people are reacting to. But I want to note that IMO the question is a vastly more helpful line of inquiry than “What justifies privileging this hypothesis over all others we could consider instead?” The answer to that question is, “I noticed it and it stands out as relevant.” And that’s significant information. But it’s also info you didn’t need to ask me for: you had it already because you could see that I put the hypothesis forward.
Now, you mention that you “can’t think of a reason” to privilege the trauma hypothesis. And yet I gave a post full of reasons.
You describe a pattern in the community, and then you ask if the dire predictions of AI 2027, specifically, are distorted by that pattern. I think that’s what I’m getting stuck on.
Imagine if the AI Futures team had nothing to do with the rationality community, but they’d made the same dire predictions. Then 2028 rolls around, and we don’t seem to be any closer to doom. People ask why AI 2027 was wrong. Would you still point to trauma-leading-to-doom-fixation as a hypothesis? (Or, rather, would you point to it just as readily?)
You describe a pattern in the community, and then you ask if the dire predictions of AI 2027, specifically, are distorted by that pattern.
Er, I don’t think that’s quite what I meant to say. I think it’s close, but this frame feels like it’s requiring me to take a stand I don’t care that much about. Let me see if I can clarify a bit.
My sense is more like, I see a pattern in the community. And I think the inclination to spell out AI 2027 and spread it came from that place. I thought that since AI 2027 was spelling out a pretty specific family of scenarios and in January 2028 we’d be able to tell to what extent those specific scenarios had in fact played out, we could use January 2028 to seriously re-evaluate what’s generating these predictions.
I’m now learning that the situation is more messy than that, and it’s much harder to trigger a “halt, melt, and catch fire” event with regard to producing dire predictions. Alas. I’m hoping we can still do something like it, but it’s not as clean and clear as I’d originally hoped.
Hopefully that clears up what I was saying and trying to do.
Now, you ask:
Imagine if the AI Futures team had nothing to do with the rationality community, but they’d made the same dire predictions. Then 2028 rolls around, and we don’t seem to be any closer to doom. People ask why AI 2027 was wrong. Would you still point to trauma-leading-to-doom-fixation as a hypothesis? (Or, rather, would you point to it just as readily?)
I’m afraid I don’t know what “the AI Futures team” is. So I can’t really answer this question in full honesty.
But with that caveat… I think my answer is “Yes.”
I’m not that picky about which subset of the community is making dire predictions and ringing the doom bell. I’m more attending to the phenomenon that doomy predictions seem to get a memetic fitness boost purely by being doomy. There are other constraints, because this community really does care about truth-tracking. But I still see a pretty non-truth-tracking incentive on memetic evolution here, and I think it’s along an axis that rationalists in general have tended to underestimate the impact of IME.
The trauma thing is a specific gearsy model of how doominess could be a memetic fitness boost independent of truth-tracking. But I’m not that picky about specifically the trauma thing being right. I just don’t know of another good gearsy model, and the trauma thing sure looks plausible to me.
I’m not that picky about which subset of the community is making dire predictions and ringing the doom bell.
My question was more like this: What if the authors weren’t a subset of the community at all? What if they’d never heard of LessWrong, somehow? Then, if the authors’ predictions turned out to be all wrong, any pattern in this community wouldn’t be the reason why; at least, that would seem pretty arbitrary to me. In reality, the authors are part of this community, and that is relevant (if I’m understanding correctly). I didn’t think about that at first; hence the question, to confirm.
Is that a good-faith response to your question?
Definitely good-faith, and the post as a whole answers more than enough.
It would be consistent with the pattern that short-timeline doomy predictions get signal-boosted here, and wouldn’t rule out the something-about-trauma-on-LessWrong hypothesis for that signal-boosting. No doubt about that! But I wasn’t talking about which predictions get signal-boosted; I was talking about which predictions get made, and in particular why the predictions in AI 2027 were made.
Consider Jane McKindaNormal, who has never heard of LessWrong and isn’t really part of the cluster at all. I wouldn’t guess that a widespread pattern among LessWrong users had affected Jane’s predictions regarding AI progress. (Eh, not directly, at least...) If Jane were the sole author of AI 2027, I wouldn’t guess that she’s making short-timeline doomy predictions because people are doing so on LessWrong. If all of her predictions were wrong, I wouldn’t guess that she mispredicted because of something-about-trauma-on-LessWrong. Perhaps she could have mispredicted because of something-about-trauma-by-herself, but there are a lot of other hypotheses hanging around, and I wouldn’t start with the hard-to-falsify ones about her upbringing.
I realized, after some thought, that the AI 2027 authors are part of the cluster, and I hadn’t taken that into account. “Oh, that might be it,” I thought. “OP is saying that we should (prepare to) ask if Kokotajlo et al, specifically, have preverbal trauma that influenced their timeline forecasts. That seemed bizarre to me at first, but it makes some sense to ask that because they’re part of the LW neighborhood, where other people are showing signs of the same thing. We wouldn’t ask this about Jane McKindaNormal.” Hence the question, to make sure that I had figured out my mistake. But it looks like I was still wrong. Now my thoughts are more like, “Eh, looks like I was focusing too hard on a few sentences and misinterpreting them. The OP is less focused on why some people have short timelines, and more on how those timelines get signal-boosted while others don’t.” (Maybe that’s still not exactly right, though.)
Okay, at the risk of starting a big side argument that’s not cruxy to the OP, I’m going to push for something controversial here:
I think it’s a mistake to view privileging the hypothesis as a fallacy.
Eliezer gives a fictional example like so:
Yes, nakedly without any context whatsoever, this is dumb. Why consider Mr. Snodgrass?
But with some context, there absolutely is a reason:
There’s some cause to why the detective suggested Mr. Snodgrass in particular.
It’s not just that there might be some cause. There must be.
Now, there are lots of causes we might not want to allow:
Maybe he had a random number generator pick out a random person and it happend to be Mr. Snodgrass.
Maybe he personally hates Mr. Snodgrass and sees an opportunity to skewer him.
Maybe he knows who did commit the murder and is trying to cover for the real murderer, and can tell that maybe Mr. Snodgrass could be framed for it.
But with the exception of the random number generator, there’s necessarily information entangled with the choice of poor Mortimer.
Any choice of a hypothesis necessarily must be kind of arbitrary. John Vervaeke talks about this: he calls it “relevance realization”. It’s a pragmatically (if not theoretically) incomputable problem: how do you decide what parts of reality are relevant? As far as I know, all living systems come up with a partial functional answer based on just grabbing some starting point and then improvising, plus adding a bit of random variance so as to catch potentially relevant things that would otherwise be filtered out.
So if you add a caveat of “But wait, this is privileging the hypothesis!”, you’re functionally putting sand in the gears of exploration. I think it’s an anti-pattern.
An adjacent and much more helpful process is “Huh, what raised this hypothesis to consideration?” Quite often the answer is “I don’t know.” But having that in mind is helpful! It lets you correct for ways in which the hypothesis can take on a different weight than it should. E.g., it helps a lot to know the detective used a random number generator to pick out Mr. Snodgrass.
(That particular case also makes the guess useless because that random number contains no information — whereas the detective’s personal hatred of Mr. Snodgrass does contain information!)
Now, you mention that you “can’t think of a reason” to privilege the trauma hypothesis. And yet I gave a post full of reasons. Is it that you needed some reason before what I said to consider it? If so, I think that’s the “privileging the hypothesis fallacy” fallacy. With what methods would you pick out a hypothesis worth considering? Why are you privileging those methods; what justifies them as hypothesis-pickers? What justifies those answers for your meta-hypothesis pickers? Etc. It’s not a coherent starting point for a functional epistemology that a human can actually do AFAICT.
The problem that privileging the hypothesis is pointing out is when someone fixates on a possibility and then uses that fixation to filter out or significantly bias evidence. But that’s not what’s happening here. I’m saying “Hey, I keep seeing a pattern that sure looks like it could be X, and if so then there’s a real and relevant problem being neglected. Here’s a suggestion about a way to check whether it’s X. Does that check work?” That’s absolutely not the kind of fixation that Eliezer was warning about!
There is, however, a different question that strikes me as relevant (!): “What caused you, Val, to point at this trauma model in particular? Why are you looking at this possibility in particular? Why this X?”
I don’t want to try to add an answer to that question in this comment. I think that’ll create noise as to what people are reacting to. But I want to note that IMO the question is a vastly more helpful line of inquiry than “What justifies privileging this hypothesis over all others we could consider instead?” The answer to that question is, “I noticed it and it stands out as relevant.” And that’s significant information. But it’s also info you didn’t need to ask me for: you had it already because you could see that I put the hypothesis forward.
Thanks for the thorough explanation.
You describe a pattern in the community, and then you ask if the dire predictions of AI 2027, specifically, are distorted by that pattern. I think that’s what I’m getting stuck on.
Imagine if the AI Futures team had nothing to do with the rationality community, but they’d made the same dire predictions. Then 2028 rolls around, and we don’t seem to be any closer to doom. People ask why AI 2027 was wrong. Would you still point to trauma-leading-to-doom-fixation as a hypothesis? (Or, rather, would you point to it just as readily?)
Er, I don’t think that’s quite what I meant to say. I think it’s close, but this frame feels like it’s requiring me to take a stand I don’t care that much about. Let me see if I can clarify a bit.
My sense is more like, I see a pattern in the community. And I think the inclination to spell out AI 2027 and spread it came from that place. I thought that since AI 2027 was spelling out a pretty specific family of scenarios and in January 2028 we’d be able to tell to what extent those specific scenarios had in fact played out, we could use January 2028 to seriously re-evaluate what’s generating these predictions.
I’m now learning that the situation is more messy than that, and it’s much harder to trigger a “halt, melt, and catch fire” event with regard to producing dire predictions. Alas. I’m hoping we can still do something like it, but it’s not as clean and clear as I’d originally hoped.
Hopefully that clears up what I was saying and trying to do.
Now, you ask:
I’m afraid I don’t know what “the AI Futures team” is. So I can’t really answer this question in full honesty.
But with that caveat… I think my answer is “Yes.”
I’m not that picky about which subset of the community is making dire predictions and ringing the doom bell. I’m more attending to the phenomenon that doomy predictions seem to get a memetic fitness boost purely by being doomy. There are other constraints, because this community really does care about truth-tracking. But I still see a pretty non-truth-tracking incentive on memetic evolution here, and I think it’s along an axis that rationalists in general have tended to underestimate the impact of IME.
The trauma thing is a specific gearsy model of how doominess could be a memetic fitness boost independent of truth-tracking. But I’m not that picky about specifically the trauma thing being right. I just don’t know of another good gearsy model, and the trauma thing sure looks plausible to me.
Is that a good-faith response to your question?
I mean the authors of AI 2027. The AI Futures Project, I should have said.
My question was more like this: What if the authors weren’t a subset of the community at all? What if they’d never heard of LessWrong, somehow? Then, if the authors’ predictions turned out to be all wrong, any pattern in this community wouldn’t be the reason why; at least, that would seem pretty arbitrary to me. In reality, the authors are part of this community, and that is relevant (if I’m understanding correctly). I didn’t think about that at first; hence the question, to confirm.
Definitely good-faith, and the post as a whole answers more than enough.
Wouldn’t that not change it very much, because the community signal-boosting a claim from outside the community still fits the pattern?
It would be consistent with the pattern that short-timeline doomy predictions get signal-boosted here, and wouldn’t rule out the something-about-trauma-on-LessWrong hypothesis for that signal-boosting. No doubt about that! But I wasn’t talking about which predictions get signal-boosted; I was talking about which predictions get made, and in particular why the predictions in AI 2027 were made.
Consider Jane McKindaNormal, who has never heard of LessWrong and isn’t really part of the cluster at all. I wouldn’t guess that a widespread pattern among LessWrong users had affected Jane’s predictions regarding AI progress. (Eh, not directly, at least...) If Jane were the sole author of AI 2027, I wouldn’t guess that she’s making short-timeline doomy predictions because people are doing so on LessWrong. If all of her predictions were wrong, I wouldn’t guess that she mispredicted because of something-about-trauma-on-LessWrong. Perhaps she could have mispredicted because of something-about-trauma-by-herself, but there are a lot of other hypotheses hanging around, and I wouldn’t start with the hard-to-falsify ones about her upbringing.
I realized, after some thought, that the AI 2027 authors are part of the cluster, and I hadn’t taken that into account. “Oh, that might be it,” I thought. “OP is saying that we should (prepare to) ask if Kokotajlo et al, specifically, have preverbal trauma that influenced their timeline forecasts. That seemed bizarre to me at first, but it makes some sense to ask that because they’re part of the LW neighborhood, where other people are showing signs of the same thing. We wouldn’t ask this about Jane McKindaNormal.” Hence the question, to make sure that I had figured out my mistake. But it looks like I was still wrong. Now my thoughts are more like, “Eh, looks like I was focusing too hard on a few sentences and misinterpreting them. The OP is less focused on why some people have short timelines, and more on how those timelines get signal-boosted while others don’t.” (Maybe that’s still not exactly right, though.)