1) A situation with AIs whose intelligence is between village idiot and Einstein—assuming there is a scale to make “between” a non-poetic concept—is not very likely and probably short-lived if it does occur (unless perhaps it is engineered that way on purpose).
2) Aspects of human cognition—our particular emotions, our language forms, perhaps even pervasive mental tricks like reasoning by analogy—may be irrelevant to Optimization Processes in general, making their focus for AI research possibly “voodoo doll” methodology. AI may only deal with such things as part of communicating with humans, though mastering them well enough to participate effectively in human culture may be as difficult as inventing new technologies.
3) Optimization Processes built by Intelligent Designers can develop in ways that those built by evolution cannot because of multiple coordinated changes (this point has been beaten to death by now I think).
4) Sex is interesting.
For once, I have no complaints. I assume the path is being cleared for a discussion of what actually IS required for an optimization process to do what we need it to do (model the world, improve itself, etc), which seems only marginally related to what our brains do. If that’s where this is headed, I’m looking forward to it.
Lessons:
1) A situation with AIs whose intelligence is between village idiot and Einstein—assuming there is a scale to make “between” a non-poetic concept—is not very likely and probably short-lived if it does occur (unless perhaps it is engineered that way on purpose).
2) Aspects of human cognition—our particular emotions, our language forms, perhaps even pervasive mental tricks like reasoning by analogy—may be irrelevant to Optimization Processes in general, making their focus for AI research possibly “voodoo doll” methodology. AI may only deal with such things as part of communicating with humans, though mastering them well enough to participate effectively in human culture may be as difficult as inventing new technologies.
3) Optimization Processes built by Intelligent Designers can develop in ways that those built by evolution cannot because of multiple coordinated changes (this point has been beaten to death by now I think).
4) Sex is interesting.
For once, I have no complaints. I assume the path is being cleared for a discussion of what actually IS required for an optimization process to do what we need it to do (model the world, improve itself, etc), which seems only marginally related to what our brains do. If that’s where this is headed, I’m looking forward to it.