Yeah it’s unclear when alpha would begin to significantly decay, but current (human) hedge funds worldwide find quite a lot. It certainly can’t continue completely indefinitely, but the ceiling seems like it would be justifiably beyond all trading revenues today.
Patrick McElroy
Karma: 33
A decision theoretic case for AI niceness
Trading firms could control meaningful amounts of compute by 2030
Apologies if this is in the post, but do you have an updated estimate for where we will be at by EoY or by 2030 based on updated assumptions post-Astra? If this would motivate a substantial downward revision in the doubling time for no CoT reasoning, I’m worried we may see 50% success no CoT horizon lengths closer to a couple hours this time next year. That would be really bad! I’m not sure how seriously to take that possibility.
Thanks for doing this work!
Great essay! I’m worried about over focusing on hardening research, but I think the same could be true for a lot of governance research. We may be able to have AI create game-theoretically perfect international agreements (or better than anything that could be worked on now) on verification or second-strike assurances, but the bottleneck to implementing them really seems like that “taste and political legitimacy” step. It seems to me that there’s very few entrenched powerful human institutions that would be open to this kind of disempowerment and any incubated one would lack legitimacy.
This is not a critique of most of these ideas, I agree with basically everything here. More curious from your point of view on recommendations, do you think research will be a bottleneck with models advanced enough for broad offensive technological acceleration?
(taking some ideas from this piece: https://www.lesswrong.com/posts/EexsebbYhbe2gXkPP/the-current-bottleneck-is-political-will-not-research)