Well, with the tool, the problem as stated only requires the last of those. If an understanding of me is required in order to achieve a test condition, the capacity for it is assumed as long as the test condition is well-defined… right? I don’t have to teach this thing the rules of Go or programming or me, as long as I can give it a test condition expressed concretely in terms of those things.
This is, admittedly, something of an abuse of your hypothetical.
Regardless, I agree that even just the last of those is a pretty tall order… as you say, not least because of the blind-optimization problem.
Then again, I was fairly dissatisfied with the Fun Theory sequence, so perhaps the exercise of putting together a set of measures that reflect my values nevertheless is an exercise worth doing, even without your tool (which does rather lower the stakes).
Well, with the tool, the problem as stated only requires the last of those. If an understanding of me is required in order to achieve a test condition, the capacity for it is assumed as long as the test condition is well-defined… right? I don’t have to teach this thing the rules of Go or programming or me, as long as I can give it a test condition expressed concretely in terms of those things.
This is, admittedly, something of an abuse of your hypothetical.
Regardless, I agree that even just the last of those is a pretty tall order… as you say, not least because of the blind-optimization problem.
Then again, I was fairly dissatisfied with the Fun Theory sequence, so perhaps the exercise of putting together a set of measures that reflect my values nevertheless is an exercise worth doing, even without your tool (which does rather lower the stakes).