I just want to be annoying and drop a “hey, don’t judge a book by it’s cover!”
There might be deeper modelling concerns that we’ve got no clue about, it’s weird and is a negative signal but it is often very hard to see second order consequences and similar from a distance!
(I literally know nothing about this situation but I just want to point it out)
I don’t know the situation, but over the past couple years I’ve updated significantly in the direction of “if an AI safety person is doing something that looks like it’s increasing x-risk, it’s not because there’s some secret reason why it’s good actually; it’s because they’re increasing x-risk.” Without knowing the details, my prior is a 75% chance that that’s what’s going on here.
The “Situational Awareness” essays downplayed AI risk and encouraged an arms race.
This is evidence that the investors in Aschenbrenner’s fund will probably also be the sorts of people who don’t care about AI risk.
It also personally makes Aschenbrenner wealthier, which gives him more power to do more things like downplaying AI risk and encouraging an arms race.
Investing in AI companies gives them more capital with which to build dangerous AI. I’m not super concerned about this for public companies at smaller scales, but I think it’s a bigger deal for private companies, where funding matters more; I don’t know if the Situational Awareness fund invests in private companies. The fund is also pretty big so it may be meaningfully accelerating timelines via investing in public companies; pretty hard to say how big that effect is.
I just want to be annoying and drop a “hey, don’t judge a book by it’s cover!”
There might be deeper modelling concerns that we’ve got no clue about, it’s weird and is a negative signal but it is often very hard to see second order consequences and similar from a distance!
(I literally know nothing about this situation but I just want to point it out)
I don’t know the situation, but over the past couple years I’ve updated significantly in the direction of “if an AI safety person is doing something that looks like it’s increasing x-risk, it’s not because there’s some secret reason why it’s good actually; it’s because they’re increasing x-risk.” Without knowing the details, my prior is a 75% chance that that’s what’s going on here.
Why do you think this hedge fund is increasing AI risk?
Mostly indirect evidence:
The “Situational Awareness” essays downplayed AI risk and encouraged an arms race.
This is evidence that the investors in Aschenbrenner’s fund will probably also be the sorts of people who don’t care about AI risk.
It also personally makes Aschenbrenner wealthier, which gives him more power to do more things like downplaying AI risk and encouraging an arms race.
Investing in AI companies gives them more capital with which to build dangerous AI. I’m not super concerned about this for public companies at smaller scales, but I think it’s a bigger deal for private companies, where funding matters more; I don’t know if the Situational Awareness fund invests in private companies. The fund is also pretty big so it may be meaningfully accelerating timelines via investing in public companies; pretty hard to say how big that effect is.
This all just seems extremely weak to me.