I am trying to develop a function that preserves human existence. You are objecting that you might stub your toe. I think we may be speaking past each other.
I feel that a good FAI designer shouldn’t dismiss these objections so easily. The toe-stubbing problem is much simpler than preserving human existence. To solve the toe-stubbing problem, you just need to correctly order two outcomes A and B. To preserve human existence, you need to order outcome A against all possible outcomes B that a powerful creative intelligence might come up with. Any solution to the bigger problem that doesn’t allow you to solve smaller problems is very suspicious to me.
I guess I disagree with this assessment of which problem is easier.
Humanity continues to exist while people stub their toes all the time. I.e., currently the humanity existing problem is close to solved, and the toe stubbing problem has by and large not been.
I am trying to develop a function that preserves human existence. You are objecting that you might stub your toe. I think we may be speaking past each other.
I feel that a good FAI designer shouldn’t dismiss these objections so easily. The toe-stubbing problem is much simpler than preserving human existence. To solve the toe-stubbing problem, you just need to correctly order two outcomes A and B. To preserve human existence, you need to order outcome A against all possible outcomes B that a powerful creative intelligence might come up with. Any solution to the bigger problem that doesn’t allow you to solve smaller problems is very suspicious to me.
I guess I disagree with this assessment of which problem is easier.
Humanity continues to exist while people stub their toes all the time. I.e., currently the humanity existing problem is close to solved, and the toe stubbing problem has by and large not been.