OK again I’m a beginner here so pls correct me, I’d be grateful:
I would offer that any set of goals given to this AGI would include the safety-concerns of humans. (Is this controversial?) Not theoretical intelligence for a thesis, but AGI acting in the world with the ability to affect us. Because of the nature of our goals, it doesn’t even seem logical to say that the AGI has gained more intelligence without also gaining an equal amount of safety-consciousness.
e.g. it’s either getting better at safely navigating the highway, or it’s still incompetent at driving.
Out on a limb: Further, bc orthogonality seems to force the separation between safety and competency, you have EY writing various intense treatises in the vain hopes that FAIR / etc will merely pay attention to safety-concerns. This just seems ridiculous, so there must be a reason, and my wild theory is that Orthogonality provides the cover needed to charge ahead with a nuke you can’t steer—but it sure goes farther and faster every month, doesn’t it?
(Now I’m guessing, and this can’t be true, but then again, why would EY say what he said about FAIR?) But they go on their merry-way because they think, “the AI is increasingly competent...no need to concerns ourselves with ‘orthogonal’ issues like safety”.
Respectfully,
Eugene
Why does EY bring up “orthogonality” so early, and strongly (“in denial”, “and why they’re true”) ? Why does it seem so important that it be accepted? thanks!