Machine intelligence will likely have an extended genesis at the hands of humanity—and during its symbiosis with us, there will be a lot of time for us to imprint our values on it.
Indeed, some would say this process has already started. Governments are likely to become superintelligent agents in the future—and they already have detailed and elaborate codifications of the things that many humans value negatively—in the form of their legal systems.
That sounds like the SIAI party line :-(
Machine intelligence will likely have an extended genesis at the hands of humanity—and during its symbiosis with us, there will be a lot of time for us to imprint our values on it.
Indeed, some would say this process has already started. Governments are likely to become superintelligent agents in the future—and they already have detailed and elaborate codifications of the things that many humans value negatively—in the form of their legal systems.