There’s evidence among humans that problem solving can be decoupled from having a functioning long-term goal system (i.e., autistic savants, certain types of frontal lobe damage, etc). So I think it’s possible to create a ‘dumb’ problem solver that won’t form any nefarious goals (or any goals) and would be overseen by humans. Furthermore, I think Nesov is wrong here; creating robust institutions and social stability, curing disease, etc—the elements of “saving the world”—fall under technical problem solving just as much as uploading, nano tech, protein folding or whatever. Advanced tech is inclusive of world saving.
There’s evidence among humans that problem solving can be decoupled from having a functioning long-term goal system (i.e., autistic savants, certain types of frontal lobe damage, etc). So I think it’s possible to create a ‘dumb’ problem solver that won’t form any nefarious goals (or any goals) and would be overseen by humans. Furthermore, I think Nesov is wrong here; creating robust institutions and social stability, curing disease, etc—the elements of “saving the world”—fall under technical problem solving just as much as uploading, nano tech, protein folding or whatever. Advanced tech is inclusive of world saving.