To make this work, the demons will need to think in a particular mathematical language, whose primitives they take for granted and have relatively unmysterious empirical significance.
Alternatively, perhaps demons are somehow forced (or motivated) to ‘do what i mean’—they never perversely interpret the semantics of what you say. But they also don’t coherently extrapolate your volition, which means they’re free to perversely manipulate aspects of the situation that you didn’t explicitly talk about (especially when you didn’t consciously think about them either). E.g., if you give a demon the English-language instruction “pick up that bucket of water,” it isn’t free to come up with a suboptimal semantics (like “pick up” means “decapitate” and “bucket of water” means “all my friends”), but it is free to execute the correctly-interpreted instruction in a dangerous way (e.g., picking it up with so much speed and force that it produces a shockwave). If demons interpret the meaning of commands with maximal benevolence but choose a means to the specified end with maximal malice, then mathematically specifying everything about the means makes demons safe(r).
To make this work, the demons will need to think in a particular mathematical language, whose primitives they take for granted and have relatively unmysterious empirical significance.
Alternatively, perhaps demons are somehow forced (or motivated) to ‘do what i mean’—they never perversely interpret the semantics of what you say. But they also don’t coherently extrapolate your volition, which means they’re free to perversely manipulate aspects of the situation that you didn’t explicitly talk about (especially when you didn’t consciously think about them either). E.g., if you give a demon the English-language instruction “pick up that bucket of water,” it isn’t free to come up with a suboptimal semantics (like “pick up” means “decapitate” and “bucket of water” means “all my friends”), but it is free to execute the correctly-interpreted instruction in a dangerous way (e.g., picking it up with so much speed and force that it produces a shockwave). If demons interpret the meaning of commands with maximal benevolence but choose a means to the specified end with maximal malice, then mathematically specifying everything about the means makes demons safe(r).