In my view, Ben reserves the right to not make sense. This might have advantages. He doesn’t have to fool himself as much as someone who believes they’re approximating True Rationality. Maybe it makes him more creative. Maybe it helps distinguish himself socially (to have more fun with more people).
Ben might have an invisible dragon in his garage (Psi). There’s no reason to rule out any possibility, especially if you believe universe=simulation is a real possibility, but he seems be hinting in belief in something specific. But this doesn’t mean whatever he says in other areas is useless.
Ben isn’t even sure that the type of unfriendly AI that has some scary fixed goal is possible. But general-drive-satisfying AI like his research is possible. He sticks by his belief that paperclip-tiling types goals really are “stupid” in some sense that should relieve us from worrying about the possibility of powerful intelligent agents having such goals. He misses the mark by not thinking about the fact that the improbability of one particular scary-arbitrary goal in no way means that the category (goals we wouldn’t be pleased with in our accidentally released unbeatable AI) has no measure.
In my view, Ben reserves the right to not make sense. This might have advantages. He doesn’t have to fool himself as much as someone who believes they’re approximating True Rationality. Maybe it makes him more creative. Maybe it helps distinguish himself socially (to have more fun with more people).
Ben might have an invisible dragon in his garage (Psi). There’s no reason to rule out any possibility, especially if you believe universe=simulation is a real possibility, but he seems be hinting in belief in something specific. But this doesn’t mean whatever he says in other areas is useless.
Ben isn’t even sure that the type of unfriendly AI that has some scary fixed goal is possible. But general-drive-satisfying AI like his research is possible. He sticks by his belief that paperclip-tiling types goals really are “stupid” in some sense that should relieve us from worrying about the possibility of powerful intelligent agents having such goals. He misses the mark by not thinking about the fact that the improbability of one particular scary-arbitrary goal in no way means that the category (goals we wouldn’t be pleased with in our accidentally released unbeatable AI) has no measure.