(A) a genie that’s going to be “just be a dick” but is not skilled at it ;
(B) a genie that is benevolent ;
(C) a genie that’s going to be “just be a dick” but is very skilled at it.
Your test will (may at least) tell apart A from (B or C). It won’t tell apart B from C.
The “there is no safe wish” rule applies to C. Sure, if your genie is not skilled a being “evil” (having an utility function very different from yours), you can craft a wish that is beyond the genie’s ability to twist it. But if the genie is skilled, much more intelligent than you are, with like the ability to spend the equivalent of one million of years of thinking how to twist the wish in one second, he’ll find a flaw and use it.
If you consider three class of genies :
(A) a genie that’s going to be “just be a dick” but is not skilled at it ;
(B) a genie that is benevolent ;
(C) a genie that’s going to be “just be a dick” but is very skilled at it.
Your test will (may at least) tell apart A from (B or C). It won’t tell apart B from C.
The “there is no safe wish” rule applies to C. Sure, if your genie is not skilled a being “evil” (having an utility function very different from yours), you can craft a wish that is beyond the genie’s ability to twist it. But if the genie is skilled, much more intelligent than you are, with like the ability to spend the equivalent of one million of years of thinking how to twist the wish in one second, he’ll find a flaw and use it.