I remember the first time I met Leopold Aschenbrenner. He questioned me about why my timelines weren’t longer and finally said, “Why aren’t you updating at all in the direction of all the smart people saying longer timelines?” (aka: OpenPhil doctrine of 2050), “At this point you’re not even pretending to be rational.” I hesitated briefly, because I knew it would not end well to answer honestly, but there wasn’t any good ending past there; and then I told him honestly that I did not consider those people to be my epistemic peers.
I don’t usually consider myself good at reading faces, but wow, the sheer LOOK of contempt and scorn that crossed his face; before he turned away and walked off without another word, dismissing me from further consideration. I will never forget that moment—because of it being such a LOUD face and my actually being able to read it, not because it had any influence on my estimates, of course.
Amusingly, I have later heard from OpenPhil people telling me very earnestly that they think OpenPhil avoided groupthink on AGI timelines because there was dissent allowed inside OpenPhil. Well, sure there was allowed a pretense of dissent inside the OpenPhil Overton window, unless you said that you weren’t updating your opinion in favor of deferring to the people whom OpenPhil considered locally high-status enough that deferring to them was rational, and then you would get frozen out and dismissed from all consideration. I can only imagine the Look that Aschenbrenner would have given somebody who’d said the same thing while being an intern rather than Yudkowsky, and while other OpenPhil personnel might’ve been more polite, I don’t really see that intern being hired.
OpenPhil and EAs may consider themselves to be terribly modest and outside-viewing, but of course that’s all an utter sham given their actual skill levels, and they have no concept of what it would actually look like for them to be humble.
It doesn’t surprise me in the least what Aschenbrenner went on to do. But yes, he is definitely a “Modesty!! Outside view!!!” sort of person, or knows that is what he is supposed to perform. I don’t expect there will ever be any contradiction that he understands, between “modesty”, and whatever the hell thought he decides to take into his head. That would take skill, and he is not aware that he is unskilled.
Luke Mueuhlhauser is actually modest. I don’t think he would ever blow up a hedge fund. It also didn’t surprise me at all when he moved on to OpenPhil, and I was glad for him, because I knew MIRI had not fit him and he would be happier at OpenPhil, and Luke had worked hard at MIRI and maybe saved the organization while he lasted. In related news, EAs picked Aschenbrenner rather than Mueuhlhauser to run their hedge fund.
EAs picked Aschenbrenner rather than Mueuhlhauser to run their hedge fund.
This sounds like it’s making a bunch of assumptions that seem wrong to me.
My guess is that Aschenbrenner wanted to found that hedge fund and that this wasn’t rooted in any sort of EA consensus that it was a good use of his time.
My guess is there wasn’t, at the time, any sort of general opinion that founding that hedge fund was a particularly more important or central position than doing AI governance grantmaking at OpenPhil. (As Mueuhlhauser was picked to do.)
Some EAs did choose to invest in the hedge fund (though my guess would’ve been it’s a minority of the fund’s money, and wasn’t necessary to get it off the ground). It still looks like the fund has made its investors lots of money (even post-crash it’s up like 80% this year? and great returns before 2026), so doesn’t seem like we need to invoke any special kind of irrationality to explain that decision.
Some EAs did choose to invest in the hedge fund (though my guess would’ve been it’s a minority of the fund’s money, and wasn’t necessary to get it off the ground). It still looks like the fund has made its investors lots of money (even post-crash it’s up like 80% this year? and great returns before 2026), so doesn’t seem like we need to invoke any special kind of irrationality to explain that decision
To be clear, public accounting suggests the hedge fund lost ~all money that wasn’t invested in Anthropic. The only reason the fund is up is because they got in early on Anthropic with a substantial chunk of their assets (25% of their assets, at 620% returns, with the rest going to zero is what seems to best produce the net-80% number).
I think the fund is indeed largely a failure, as of course basically anyone investing in the hedge fund would have loved to also invest in Anthropic. In that sense, the fund maybe provided some value by being a vehicle for investing early, but that could have also been achieved many other ways, and clearly isn’t why people were investing. If you remove that share, the fund went approximately completely bust.
(I originally had the same impression as you had, which is bad! I think the shareholder letter and public statements by Leopold have been pretty misleading here, in the sense that basically everyone I have talked who learned the actual details of what happened updated substantially negatively after first reading the shareholder letter)
I don’t usually consider myself good at reading faces, but wow, the sheer LOOK of contempt and scorn that crossed his face; mefore he turned away and walked off without another word, forever dismissing me from his consideration. I will never forget that moment
A bit off topic, but lately I’ve been thinking about the power of grudges. It’s just so underestimated. There are people who did a minor bad thing to me once (on the order of, make a noise complaint against me) and I kinda hate them forever, even years later. And then I think of people I myself might’ve offended in minor ways in my life, and there’s so many, ouch.
I haven’t found any easy way to fight it, but one counterweight that I’ve kinda stumbled onto is holding “positive grudges”. When someone does me a minor good thing, I make a point of remembering it forever and expressing it when I can. It’s a bit weird, but it makes me feel good.
I rather expected from that time that Aschenbrenner would be trouble for EA, if they just couldn’t stop themselves from promoting Harvard blondes; but I did not say anything against him then on the basis of that conversation, and only marked him down as actively harmful once I saw him starting to oppose all good ideas during his brief tenure at FTX philanthropy. He did of course further go on to actively stoke arms races with China.
The relevance to the Wei Dai part is that Wei Dai seemed to me to be asking, “Aren’t you deciding post facto who is ‘EA’ and who is ‘rationalist’?” and my reply to Wei Dai is implicitly, “My memory claims that it is really really not hard to tell, in advance, early on.”
I had a somewhat similar though shorter interaction with Jaime Sevilla, now leading Epoch. Raised short timelines in around 2021 hoping for a double crux, got look of dismissive contempt and disengagement. Kinda wish I had been more public about my read of them given how they ended incubating a capabilities startup.
I remember the first time I met Leopold Aschenbrenner. He questioned me about why my timelines weren’t longer and finally said, “Why aren’t you updating at all in the direction of all the smart people saying longer timelines?” (aka: OpenPhil doctrine of 2050), “At this point you’re not even pretending to be rational.” I hesitated briefly, because I knew it would not end well to answer honestly, but there wasn’t any good ending past there; and then I told him honestly that I did not consider those people to be my epistemic peers.
I don’t usually consider myself good at reading faces, but wow, the sheer LOOK of contempt and scorn that crossed his face; before he turned away and walked off without another word, dismissing me from further consideration. I will never forget that moment—because of it being such a LOUD face and my actually being able to read it, not because it had any influence on my estimates, of course.
Amusingly, I have later heard from OpenPhil people telling me very earnestly that they think OpenPhil avoided groupthink on AGI timelines because there was dissent allowed inside OpenPhil. Well, sure there was allowed a pretense of dissent inside the OpenPhil Overton window, unless you said that you weren’t updating your opinion in favor of deferring to the people whom OpenPhil considered locally high-status enough that deferring to them was rational, and then you would get frozen out and dismissed from all consideration. I can only imagine the Look that Aschenbrenner would have given somebody who’d said the same thing while being an intern rather than Yudkowsky, and while other OpenPhil personnel might’ve been more polite, I don’t really see that intern being hired.
OpenPhil and EAs may consider themselves to be terribly modest and outside-viewing, but of course that’s all an utter sham given their actual skill levels, and they have no concept of what it would actually look like for them to be humble.
It doesn’t surprise me in the least what Aschenbrenner went on to do. But yes, he is definitely a “Modesty!! Outside view!!!” sort of person, or knows that is what he is supposed to perform. I don’t expect there will ever be any contradiction that he understands, between “modesty”, and whatever the hell thought he decides to take into his head. That would take skill, and he is not aware that he is unskilled.
Luke Mueuhlhauser is actually modest. I don’t think he would ever blow up a hedge fund. It also didn’t surprise me at all when he moved on to OpenPhil, and I was glad for him, because I knew MIRI had not fit him and he would be happier at OpenPhil, and Luke had worked hard at MIRI and maybe saved the organization while he lasted. In related news, EAs picked Aschenbrenner rather than Mueuhlhauser to run their hedge fund.
This sounds like it’s making a bunch of assumptions that seem wrong to me.
My guess is that Aschenbrenner wanted to found that hedge fund and that this wasn’t rooted in any sort of EA consensus that it was a good use of his time.
My guess is there wasn’t, at the time, any sort of general opinion that founding that hedge fund was a particularly more important or central position than doing AI governance grantmaking at OpenPhil. (As Mueuhlhauser was picked to do.)
Some EAs did choose to invest in the hedge fund (though my guess would’ve been it’s a minority of the fund’s money, and wasn’t necessary to get it off the ground). It still looks like the fund has made its investors lots of money (even post-crash it’s up like 80% this year? and great returns before 2026), so doesn’t seem like we need to invoke any special kind of irrationality to explain that decision.
To be clear, public accounting suggests the hedge fund lost ~all money that wasn’t invested in Anthropic. The only reason the fund is up is because they got in early on Anthropic with a substantial chunk of their assets (25% of their assets, at 620% returns, with the rest going to zero is what seems to best produce the net-80% number).
I think the fund is indeed largely a failure, as of course basically anyone investing in the hedge fund would have loved to also invest in Anthropic. In that sense, the fund maybe provided some value by being a vehicle for investing early, but that could have also been achieved many other ways, and clearly isn’t why people were investing. If you remove that share, the fund went approximately completely bust.
See: https://x.com/ohabryka/status/2083698812682731749
(I originally had the same impression as you had, which is bad! I think the shareholder letter and public statements by Leopold have been pretty misleading here, in the sense that basically everyone I have talked who learned the actual details of what happened updated substantially negatively after first reading the shareholder letter)
A bit off topic, but lately I’ve been thinking about the power of grudges. It’s just so underestimated. There are people who did a minor bad thing to me once (on the order of, make a noise complaint against me) and I kinda hate them forever, even years later. And then I think of people I myself might’ve offended in minor ways in my life, and there’s so many, ouch.
I haven’t found any easy way to fight it, but one counterweight that I’ve kinda stumbled onto is holding “positive grudges”. When someone does me a minor good thing, I make a point of remembering it forever and expressing it when I can. It’s a bit weird, but it makes me feel good.
I rather expected from that time that Aschenbrenner would be trouble for EA, if they just couldn’t stop themselves from promoting Harvard blondes; but I did not say anything against him then on the basis of that conversation, and only marked him down as actively harmful once I saw him starting to oppose all good ideas during his brief tenure at FTX philanthropy. He did of course further go on to actively stoke arms races with China.
The relevance to the Wei Dai part is that Wei Dai seemed to me to be asking, “Aren’t you deciding post facto who is ‘EA’ and who is ‘rationalist’?” and my reply to Wei Dai is implicitly, “My memory claims that it is really really not hard to tell, in advance, early on.”
I had a somewhat similar though shorter interaction with Jaime Sevilla, now leading Epoch. Raised short timelines in around 2021 hoping for a double crux, got look of dismissive contempt and disengagement. Kinda wish I had been more public about my read of them given how they ended incubating a capabilities startup.