I mean, maybe we should just drop this point about the intuition pump, it was a throwaway reference in the original comment. I normally use it to argue against a specific mentality I sometimes see in people, and I guess it doesn’t make sense outside of that context.
(The mentality is “it doesn’t matter what oversight process you use, there’s always a malicious superintelligence that can game it, therefore everyone dies”.)
I mean, maybe we should just drop this point about the intuition pump, it was a throwaway reference in the original comment. I normally use it to argue against a specific mentality I sometimes see in people, and I guess it doesn’t make sense outside of that context.
(The mentality is “it doesn’t matter what oversight process you use, there’s always a malicious superintelligence that can game it, therefore everyone dies”.)