Programming a computer to reliably make lots of diamonds (or paperclips) is not creating Clippy for the same reason that programming Google Maps to produce the shortest distance between two locations is not creating Clippy. People program computers to do X, where X doesn’t consider the welfare of humans, all the time. The programming is not really “do X no matter what”, it’s “do X using these methods”. Google Maps will not start trying to hack the computers of construction equipment in order to build a bridge and shorten the distance it finds between two points.
Programming a computer to reliably make lots of diamonds (or paperclips) is not creating Clippy for the same reason that programming Google Maps to produce the shortest distance between two locations is not creating Clippy.
Ok, but that makes Nate’s statement very confusing. We already understand, “up to” R&D effort, how to program computers to use various peripherals to perform a task in the physical world without intelligence, using fixed methods. I’m left confused at what industrial automation has to do with AI alignment research.
Programming a computer to reliably make lots of diamonds (or paperclips) is not creating Clippy for the same reason that programming Google Maps to produce the shortest distance between two locations is not creating Clippy. People program computers to do X, where X doesn’t consider the welfare of humans, all the time. The programming is not really “do X no matter what”, it’s “do X using these methods”. Google Maps will not start trying to hack the computers of construction equipment in order to build a bridge and shorten the distance it finds between two points.
Ok, but that makes Nate’s statement very confusing. We already understand, “up to” R&D effort, how to program computers to use various peripherals to perform a task in the physical world without intelligence, using fixed methods. I’m left confused at what industrial automation has to do with AI alignment research.