something like Cyc might be able to parse that command
Actually we might be able to explain that command to a math-only AI pretty soon. For example, in Paul’s “formal instructions” approach, you point the AI at a long bitstring generated by a human, and ask the AI to find the most likely program that generated that bitstring under the universal prior. As a result you get something functionally similar to an upload. There are many potential problems with this approach (e.g. how do you separate the human from the rest of the world?) but they seem like the sort of problems you can solve with clever tricks, not insurmountable obstacles. And there might be other ways to use math-only AIs to boost uploading, we’ve only been thinking about this for several months.
But presumably the AI’s diagnostics would eventually tell you something like “need for computing substrate 10^9 more powerful than current processor detected”, that you could have figured out for yourself.
A probabilistic answer will suffice, I think, and it doesn’t seem to require that much computing power. It’s suspicious that you can make educated guesses about what you’re going to think tomorrow, but a strong AI somehow cannot. I’d expect a strong AI to be better at answering math questions than a human.
ETA: if solving the object-level problem is a more promising approach than bootstrapping through the meta-problem, then I certainly want to believe that is the case. Right now I feel we only have a viable attack on the meta-problem. If you figure out a viable attack on the object-level problem, be sure to let us know!
Actually we might be able to explain that command to a math-only AI pretty soon. For example, in Paul’s “formal instructions” approach, you point the AI at a long bitstring generated by a human, and ask the AI to find the most likely program that generated that bitstring under the universal prior. As a result you get something functionally similar to an upload. There are many potential problems with this approach (e.g. how do you separate the human from the rest of the world?) but they seem like the sort of problems you can solve with clever tricks, not insurmountable obstacles. And there might be other ways to use math-only AIs to boost uploading, we’ve only been thinking about this for several months.
A probabilistic answer will suffice, I think, and it doesn’t seem to require that much computing power. It’s suspicious that you can make educated guesses about what you’re going to think tomorrow, but a strong AI somehow cannot. I’d expect a strong AI to be better at answering math questions than a human.
ETA: if solving the object-level problem is a more promising approach than bootstrapping through the meta-problem, then I certainly want to believe that is the case. Right now I feel we only have a viable attack on the meta-problem. If you figure out a viable attack on the object-level problem, be sure to let us know!