Hal Finney:
Why doesn’t the AI do it verself? Even if it’s boxed (and why would it be, if I’m convinced it’s an FAI?), at the intelligence it’d need to make the stated prediction with any degree of confidence, I’d expect it to be able to take over my mind quickly. If what it claims is correct, it shouldn’t have any qualms about doing that (taking over one human’s body for a few minutes is a small price to pay for the utility involved).
If this happened in practice I’d be confused as heck, and the alleged FAI being honest about its intentions would be pretty far down on my list of hypotheses about what’s going on. I’d likely stare into space dumbfounded until I found some halfway-likely explanation, or the AI decided to take over my mind after all.
Hal Finney:
Why doesn’t the AI do it verself? Even if it’s boxed (and why would it be, if I’m convinced it’s an FAI?), at the intelligence it’d need to make the stated prediction with any degree of confidence, I’d expect it to be able to take over my mind quickly. If what it claims is correct, it shouldn’t have any qualms about doing that (taking over one human’s body for a few minutes is a small price to pay for the utility involved).
If this happened in practice I’d be confused as heck, and the alleged FAI being honest about its intentions would be pretty far down on my list of hypotheses about what’s going on. I’d likely stare into space dumbfounded until I found some halfway-likely explanation, or the AI decided to take over my mind after all.