All computer programming will be performed and supervised by engineered agents eventually. But so what? That is right, natural and desirable.
It seems as though you are presuming a superintelliigence which doesn’t want to do what humans tell it to. I am sure that will be true for some humans—not everyone can apply patches to Google today. However, for other humans, the superintelligence will probably be keen to do whatever they ask of it—since it will have been built to do just that.
All computer programming will be performed and supervised by engineered agents eventually. But so what? That is right, natural and desirable.
It seems as though you are presuming a superintelliigence which doesn’t want to do what humans tell it to. I am sure that will be true for some humans—not everyone can apply patches to Google today. However, for other humans, the superintelligence will probably be keen to do whatever they ask of it—since it will have been built to do just that.