When you keep saying “it’s not a solved problem” and “no solution”, are you saying that there are important mysteries about how the human brain works, or that there are important mysteries about how to solve the technical alignment problem for brain-like AGI?
If the latter, yes I strongly agree, I say that all the time. If you think I have ever claimed to know a good plan for technical alignment (or corrigibility etc.) then either I made a typo or you misunderstood me. For example, when I wrote “the AI keeps the supervisor’s desires / preferences / aversions at the back of its mind …”, the context was a discussion of high-level desiderata / research directions, not nuts-and-bolts technical plans.
If the former (if you’re saying that there are important mysteries about how the human brain works), then I’m confused that you’re making this comment in a conversation about AI alignment, and linking to other posts on AI alignment. There’s a question of whether or not my neuro framework can explain every aspect of human psychology / behavior / experience, and that question is unrelated to AI alignment. (I obviously think the answer is yes, and if you disagree then we can talk about the aspects of human experience that you think it can’t explain, but please leave AI out of that conversation.)
I’d say “we don’t understand how the brain works good enough to replicate it in AGI” (and that means we lack a really important layer of understanding). Anyway, I think we agree here.
The point I wanted to make:
I think John’s argument for why corrigibility is natural works even if you don’t assume crisp separate modules.
Whatever explanation you propose (of the brain staying coherent), be it “your modules are corrigible to each other” or “the goals are kept at the back of your mind”, if you try to make that explanation more technical you’ll probably end up with a general solution to corrigibility.
Sorry for skipping so much inferential steps and causing confusion.
When you keep saying “it’s not a solved problem” and “no solution”, are you saying that there are important mysteries about how the human brain works, or that there are important mysteries about how to solve the technical alignment problem for brain-like AGI?
If the latter, yes I strongly agree, I say that all the time. If you think I have ever claimed to know a good plan for technical alignment (or corrigibility etc.) then either I made a typo or you misunderstood me. For example, when I wrote “the AI keeps the supervisor’s desires / preferences / aversions at the back of its mind …”, the context was a discussion of high-level desiderata / research directions, not nuts-and-bolts technical plans.
If the former (if you’re saying that there are important mysteries about how the human brain works), then I’m confused that you’re making this comment in a conversation about AI alignment, and linking to other posts on AI alignment. There’s a question of whether or not my neuro framework can explain every aspect of human psychology / behavior / experience, and that question is unrelated to AI alignment. (I obviously think the answer is yes, and if you disagree then we can talk about the aspects of human experience that you think it can’t explain, but please leave AI out of that conversation.)
I’d say “we don’t understand how the brain works good enough to replicate it in AGI” (and that means we lack a really important layer of understanding). Anyway, I think we agree here.
The point I wanted to make:
I think John’s argument for why corrigibility is natural works even if you don’t assume crisp separate modules.
Whatever explanation you propose (of the brain staying coherent), be it “your modules are corrigible to each other” or “the goals are kept at the back of your mind”, if you try to make that explanation more technical you’ll probably end up with a general solution to corrigibility.
Sorry for skipping so much inferential steps and causing confusion.