« You would very probably have to reach into the brain and rewire neural circuitry directly; I don’t think any sense input or motor interaction would accomplish such a thing. » Are you really sure it’s not possible ? To me it’s just a harder version of the AI Box problem : I’m pretty sure (> 95% probability) a sufficiently smart AI can convince anyone (who actually listens to it, at least) to let it out of the box. I’m not so sure about an AI being able to rewire my brain enough to make me an Einstein or Engelbart using only sensory inputs, but I would definitely give a significant probability (higher than 10%) than a Bayesian superintelligence could reverse-engineer the way my brain learns from stimulus enough to rewire enough parts of my brain without using any nanotechnology. Just from other humans, by reading things (like Less Wrong or many books), we can improve a lot. A superintelligence could probably do much, much more to help us improve ourselves with just normal communication.
« You would very probably have to reach into the brain and rewire neural circuitry directly; I don’t think any sense input or motor interaction would accomplish such a thing. » Are you really sure it’s not possible ? To me it’s just a harder version of the AI Box problem : I’m pretty sure (> 95% probability) a sufficiently smart AI can convince anyone (who actually listens to it, at least) to let it out of the box. I’m not so sure about an AI being able to rewire my brain enough to make me an Einstein or Engelbart using only sensory inputs, but I would definitely give a significant probability (higher than 10%) than a Bayesian superintelligence could reverse-engineer the way my brain learns from stimulus enough to rewire enough parts of my brain without using any nanotechnology. Just from other humans, by reading things (like Less Wrong or many books), we can improve a lot. A superintelligence could probably do much, much more to help us improve ourselves with just normal communication.