Given those constraints, I’d come up with the most comprehensive quality-of-life scoring system I could come up with, and ask it for a set of steps to perform that maximizes my score on that system over my lifetime.
Actually, that would probably be worth doing even without the tool.
That would involve precisely describing your existence, the possible steps you are capable of performing, and all of the quality-of-life measures you define (not to mention choosing a set of quality-of-life measures whose blind optimization is good). That seems like a pretty tall order.
Well, with the tool, the problem as stated only requires the last of those. If an understanding of me is required in order to achieve a test condition, the capacity for it is assumed as long as the test condition is well-defined… right? I don’t have to teach this thing the rules of Go or programming or me, as long as I can give it a test condition expressed concretely in terms of those things.
This is, admittedly, something of an abuse of your hypothetical.
Regardless, I agree that even just the last of those is a pretty tall order… as you say, not least because of the blind-optimization problem.
Then again, I was fairly dissatisfied with the Fun Theory sequence, so perhaps the exercise of putting together a set of measures that reflect my values nevertheless is an exercise worth doing, even without your tool (which does rather lower the stakes).
Given those constraints, I’d come up with the most comprehensive quality-of-life scoring system I could come up with, and ask it for a set of steps to perform that maximizes my score on that system over my lifetime.
Actually, that would probably be worth doing even without the tool.
That would involve precisely describing your existence, the possible steps you are capable of performing, and all of the quality-of-life measures you define (not to mention choosing a set of quality-of-life measures whose blind optimization is good). That seems like a pretty tall order.
Well, with the tool, the problem as stated only requires the last of those. If an understanding of me is required in order to achieve a test condition, the capacity for it is assumed as long as the test condition is well-defined… right? I don’t have to teach this thing the rules of Go or programming or me, as long as I can give it a test condition expressed concretely in terms of those things.
This is, admittedly, something of an abuse of your hypothetical.
Regardless, I agree that even just the last of those is a pretty tall order… as you say, not least because of the blind-optimization problem.
Then again, I was fairly dissatisfied with the Fun Theory sequence, so perhaps the exercise of putting together a set of measures that reflect my values nevertheless is an exercise worth doing, even without your tool (which does rather lower the stakes).