Capability. Superintelligence can now be developed outside of a big AI lab—via a self-improving codebase which makes thousands of recursive LLM calls.
Safety. (a) Superintelligence will become “self-interested” for some definition of self. (b) Humanity fairs well to the extent that its sense of self includes us.
Two opinions on superintelligence’s development:
Capability. Superintelligence can now be developed outside of a big AI lab—via a self-improving codebase which makes thousands of recursive LLM calls.
Safety. (a) Superintelligence will become “self-interested” for some definition of self. (b) Humanity fairs well to the extent that its sense of self includes us.