Okay, I think I can agree with you that it would have been better for our post to explicitly disclaim that if alignment is very hard, then this work is less valuable. The value of this work depends on there being some predictable link between the character A and the one we get.
(Though honestly, I think there is a decent case to be made, even from a very pessimistic perspective, for thinking more about trying to affect the world as AI does take over.)
I pretty strongly disagree with you if you think that every post on a topic like this has to be really doomy and apologetic about focusing on anything other than misalignment work or pausing AI.
On the other hand, this level of confidence would be pretty controversial even among the AI lab employees working on these techniques.
Not sure what level of confidence you’re referring to. I’m personally at about 10% for misaligned AI takeover. I don’t think the post comes across as highly confident that AI alignment will be solved. We explicitly discuss that the work can be valuable if AI does take over, acknowledging that possibility.
I pretty strongly disagree with you if you think that every post on a topic like this has to be really doomy and apologetic about focusing on anything other than misalignment work or pausing AI.
This isn’t exactly what I was trying to say. It’s a little difficult for me to try to tell you what the post “ought” to be doing, since I don’t really understand who its intended audience is, what simulacrum level it’s operating on, etc. But it feels like the analysis in the post is just incomplete, given your beliefs about (unconditional?) misalignment risk, and the level at which the post seems like it’s trying to operate[1].
Okay, I think I can agree with you that it would have been better for our post to explicitly disclaim that if alignment is very hard, then this work is less valuable. The value of this work depends on there being some predictable link between the character A and the one we get.
(Though honestly, I think there is a decent case to be made, even from a very pessimistic perspective, for thinking more about trying to affect the world as AI does take over.)
I pretty strongly disagree with you if you think that every post on a topic like this has to be really doomy and apologetic about focusing on anything other than misalignment work or pausing AI.
Not sure what level of confidence you’re referring to. I’m personally at about 10% for misaligned AI takeover. I don’t think the post comes across as highly confident that AI alignment will be solved. We explicitly discuss that the work can be valuable if AI does take over, acknowledging that possibility.
This isn’t exactly what I was trying to say. It’s a little difficult for me to try to tell you what the post “ought” to be doing, since I don’t really understand who its intended audience is, what simulacrum level it’s operating on, etc. But it feels like the analysis in the post is just incomplete, given your beliefs about (unconditional?) misalignment risk, and the level at which the post seems like it’s trying to operate[1].
Providing a high-level argument for why people should be thinking/working more on this particular subject, on the current margin.
why incomplete?