Now, you mention that you “can’t think of a reason” to privilege the trauma hypothesis. And yet I gave a post full of reasons.
You describe a pattern in the community, and then you ask if the dire predictions of AI 2027, specifically, are distorted by that pattern. I think that’s what I’m getting stuck on.
Imagine if the AI Futures team had nothing to do with the rationality community, but they’d made the same dire predictions. Then 2028 rolls around, and we don’t seem to be any closer to doom. People ask why AI 2027 was wrong. Would you still point to trauma-leading-to-doom-fixation as a hypothesis? (Or, rather, would you point to it just as readily?)
You describe a pattern in the community, and then you ask if the dire predictions of AI 2027, specifically, are distorted by that pattern.
Er, I don’t think that’s quite what I meant to say. I think it’s close, but this frame feels like it’s requiring me to take a stand I don’t care that much about. Let me see if I can clarify a bit.
My sense is more like, I see a pattern in the community. And I think the inclination to spell out AI 2027 and spread it came from that place. I thought that since AI 2027 was spelling out a pretty specific family of scenarios and in January 2028 we’d be able to tell to what extent those specific scenarios had in fact played out, we could use January 2028 to seriously re-evaluate what’s generating these predictions.
I’m now learning that the situation is more messy than that, and it’s much harder to trigger a “halt, melt, and catch fire” event with regard to producing dire predictions. Alas. I’m hoping we can still do something like it, but it’s not as clean and clear as I’d originally hoped.
Hopefully that clears up what I was saying and trying to do.
Now, you ask:
Imagine if the AI Futures team had nothing to do with the rationality community, but they’d made the same dire predictions. Then 2028 rolls around, and we don’t seem to be any closer to doom. People ask why AI 2027 was wrong. Would you still point to trauma-leading-to-doom-fixation as a hypothesis? (Or, rather, would you point to it just as readily?)
I’m afraid I don’t know what “the AI Futures team” is. So I can’t really answer this question in full honesty.
But with that caveat… I think my answer is “Yes.”
I’m not that picky about which subset of the community is making dire predictions and ringing the doom bell. I’m more attending to the phenomenon that doomy predictions seem to get a memetic fitness boost purely by being doomy. There are other constraints, because this community really does care about truth-tracking. But I still see a pretty non-truth-tracking incentive on memetic evolution here, and I think it’s along an axis that rationalists in general have tended to underestimate the impact of IME.
The trauma thing is a specific gearsy model of how doominess could be a memetic fitness boost independent of truth-tracking. But I’m not that picky about specifically the trauma thing being right. I just don’t know of another good gearsy model, and the trauma thing sure looks plausible to me.
I’m not that picky about which subset of the community is making dire predictions and ringing the doom bell.
My question was more like this: What if the authors weren’t a subset of the community at all? What if they’d never heard of LessWrong, somehow? Then, if the authors’ predictions turned out to be all wrong, any pattern in this community wouldn’t be the reason why; at least, that would seem pretty arbitrary to me. In reality, the authors are part of this community, and that is relevant (if I’m understanding correctly). I didn’t think about that at first; hence the question, to confirm.
Is that a good-faith response to your question?
Definitely good-faith, and the post as a whole answers more than enough.
It would be consistent with the pattern that short-timeline doomy predictions get signal-boosted here, and wouldn’t rule out the something-about-trauma-on-LessWrong hypothesis for that signal-boosting. No doubt about that! But I wasn’t talking about which predictions get signal-boosted; I was talking about which predictions get made, and in particular why the predictions in AI 2027 were made.
Consider Jane McKindaNormal, who has never heard of LessWrong and isn’t really part of the cluster at all. I wouldn’t guess that a widespread pattern among LessWrong users had affected Jane’s predictions regarding AI progress. (Eh, not directly, at least...) If Jane were the sole author of AI 2027, I wouldn’t guess that she’s making short-timeline doomy predictions because people are doing so on LessWrong. If all of her predictions were wrong, I wouldn’t guess that she mispredicted because of something-about-trauma-on-LessWrong. Perhaps she could have mispredicted because of something-about-trauma-by-herself, but there are a lot of other hypotheses hanging around, and I wouldn’t start with the hard-to-falsify ones about her upbringing.
I realized, after some thought, that the AI 2027 authors are part of the cluster, and I hadn’t taken that into account. “Oh, that might be it,” I thought. “OP is saying that we should (prepare to) ask if Kokotajlo et al, specifically, have preverbal trauma that influenced their timeline forecasts. That seemed bizarre to me at first, but it makes some sense to ask that because they’re part of the LW neighborhood, where other people are showing signs of the same thing. We wouldn’t ask this about Jane McKindaNormal.” Hence the question, to make sure that I had figured out my mistake. But it looks like I was still wrong. Now my thoughts are more like, “Eh, looks like I was focusing too hard on a few sentences and misinterpreting them. The OP is less focused on why some people have short timelines, and more on how those timelines get signal-boosted while others don’t.” (Maybe that’s still not exactly right, though.)
Thanks for the thorough explanation.
You describe a pattern in the community, and then you ask if the dire predictions of AI 2027, specifically, are distorted by that pattern. I think that’s what I’m getting stuck on.
Imagine if the AI Futures team had nothing to do with the rationality community, but they’d made the same dire predictions. Then 2028 rolls around, and we don’t seem to be any closer to doom. People ask why AI 2027 was wrong. Would you still point to trauma-leading-to-doom-fixation as a hypothesis? (Or, rather, would you point to it just as readily?)
Er, I don’t think that’s quite what I meant to say. I think it’s close, but this frame feels like it’s requiring me to take a stand I don’t care that much about. Let me see if I can clarify a bit.
My sense is more like, I see a pattern in the community. And I think the inclination to spell out AI 2027 and spread it came from that place. I thought that since AI 2027 was spelling out a pretty specific family of scenarios and in January 2028 we’d be able to tell to what extent those specific scenarios had in fact played out, we could use January 2028 to seriously re-evaluate what’s generating these predictions.
I’m now learning that the situation is more messy than that, and it’s much harder to trigger a “halt, melt, and catch fire” event with regard to producing dire predictions. Alas. I’m hoping we can still do something like it, but it’s not as clean and clear as I’d originally hoped.
Hopefully that clears up what I was saying and trying to do.
Now, you ask:
I’m afraid I don’t know what “the AI Futures team” is. So I can’t really answer this question in full honesty.
But with that caveat… I think my answer is “Yes.”
I’m not that picky about which subset of the community is making dire predictions and ringing the doom bell. I’m more attending to the phenomenon that doomy predictions seem to get a memetic fitness boost purely by being doomy. There are other constraints, because this community really does care about truth-tracking. But I still see a pretty non-truth-tracking incentive on memetic evolution here, and I think it’s along an axis that rationalists in general have tended to underestimate the impact of IME.
The trauma thing is a specific gearsy model of how doominess could be a memetic fitness boost independent of truth-tracking. But I’m not that picky about specifically the trauma thing being right. I just don’t know of another good gearsy model, and the trauma thing sure looks plausible to me.
Is that a good-faith response to your question?
I mean the authors of AI 2027. The AI Futures Project, I should have said.
My question was more like this: What if the authors weren’t a subset of the community at all? What if they’d never heard of LessWrong, somehow? Then, if the authors’ predictions turned out to be all wrong, any pattern in this community wouldn’t be the reason why; at least, that would seem pretty arbitrary to me. In reality, the authors are part of this community, and that is relevant (if I’m understanding correctly). I didn’t think about that at first; hence the question, to confirm.
Definitely good-faith, and the post as a whole answers more than enough.
Wouldn’t that not change it very much, because the community signal-boosting a claim from outside the community still fits the pattern?
It would be consistent with the pattern that short-timeline doomy predictions get signal-boosted here, and wouldn’t rule out the something-about-trauma-on-LessWrong hypothesis for that signal-boosting. No doubt about that! But I wasn’t talking about which predictions get signal-boosted; I was talking about which predictions get made, and in particular why the predictions in AI 2027 were made.
Consider Jane McKindaNormal, who has never heard of LessWrong and isn’t really part of the cluster at all. I wouldn’t guess that a widespread pattern among LessWrong users had affected Jane’s predictions regarding AI progress. (Eh, not directly, at least...) If Jane were the sole author of AI 2027, I wouldn’t guess that she’s making short-timeline doomy predictions because people are doing so on LessWrong. If all of her predictions were wrong, I wouldn’t guess that she mispredicted because of something-about-trauma-on-LessWrong. Perhaps she could have mispredicted because of something-about-trauma-by-herself, but there are a lot of other hypotheses hanging around, and I wouldn’t start with the hard-to-falsify ones about her upbringing.
I realized, after some thought, that the AI 2027 authors are part of the cluster, and I hadn’t taken that into account. “Oh, that might be it,” I thought. “OP is saying that we should (prepare to) ask if Kokotajlo et al, specifically, have preverbal trauma that influenced their timeline forecasts. That seemed bizarre to me at first, but it makes some sense to ask that because they’re part of the LW neighborhood, where other people are showing signs of the same thing. We wouldn’t ask this about Jane McKindaNormal.” Hence the question, to make sure that I had figured out my mistake. But it looks like I was still wrong. Now my thoughts are more like, “Eh, looks like I was focusing too hard on a few sentences and misinterpreting them. The OP is less focused on why some people have short timelines, and more on how those timelines get signal-boosted while others don’t.” (Maybe that’s still not exactly right, though.)