“ If you know anybody in politics anywhere, it might be a good idea to try and convince them to pay attention to this AGI thing”—It wouldn’t surprise me if this was net-negative and the default outcome of informing actors about AGI is for them to attempt to accelerate it.
Another part I’d disagree with is lionising technical researchers over everyone else.
The point of the post was to not lionize them over everyone else. The target audience I had in mind (which may not even exist at this point) was people who wanted to become alignment researchers because that’s where the front lines are. My point is that that may not be the best idea in some cases. At the end of the day, if we solve the alignment problem it will be directly thanks to those researchers, that’s what I mean.
As for the politics thing that’s interesting, I hadn’t thought of it horribly backfiring in that way. I mean the goal would be to explain to them why alignment is necessary, which shouldn’t be an impossible task. There’s a lot of legal and economic power coming from the government, so just ignoring that actor seems like a mistake.
Excellent post. One part I disagree with though:
“ If you know anybody in politics anywhere, it might be a good idea to try and convince them to pay attention to this AGI thing”—It wouldn’t surprise me if this was net-negative and the default outcome of informing actors about AGI is for them to attempt to accelerate it.
Another part I’d disagree with is lionising technical researchers over everyone else.
The point of the post was to not lionize them over everyone else. The target audience I had in mind (which may not even exist at this point) was people who wanted to become alignment researchers because that’s where the front lines are. My point is that that may not be the best idea in some cases. At the end of the day, if we solve the alignment problem it will be directly thanks to those researchers, that’s what I mean.
As for the politics thing that’s interesting, I hadn’t thought of it horribly backfiring in that way. I mean the goal would be to explain to them why alignment is necessary, which shouldn’t be an impossible task. There’s a lot of legal and economic power coming from the government, so just ignoring that actor seems like a mistake.
Thanks for the feedback!