Both we and the smiley-face maximizer would be better off if we did allow it to be built, and then it gives our preferences some control (enough for us to be better off than the no-AI future). It’s not clear that this opportunity for trade can be realized, but we should spend some time thinking about it before ruling it out.
Sounds like Drescher’s bounded Newcomb. This perspective suddenly painted it FAI-complete.
Sounds like Drescher’s bounded Newcomb. This perspective suddenly painted it FAI-complete.
Can you please elaborate? I looked up “FAI-complete”, and found this but I still don’t get your point.
See the DT list. (Copy of the post here.) FAI-complete problem = solving it means that FAI gets solved as well.