I’m uneasy about this for similar reasons to why I was uneasy about utility indifference. If the AI collects compelling evidence that a thermodynamic miracle did not occur, then it is possible that the hypotheses left in which the thermodynamic miracle did occur will be dominated by strange, complicated hypotheses (e.g. the existence of some sort of Cartesian demon trying to trick the AI into thinking that the thermodynamic miracle occurred), and the AI’s behavior may become erratic as a result.
Yes, this is a concern. But it seems a solvable concern, once we have the principles right (and it almost certainly won’t be implemented as an actual wire in a gas cloud setup).
I’m uneasy about this for similar reasons to why I was uneasy about utility indifference. If the AI collects compelling evidence that a thermodynamic miracle did not occur, then it is possible that the hypotheses left in which the thermodynamic miracle did occur will be dominated by strange, complicated hypotheses (e.g. the existence of some sort of Cartesian demon trying to trick the AI into thinking that the thermodynamic miracle occurred), and the AI’s behavior may become erratic as a result.
Yes, this is a concern. But it seems a solvable concern, once we have the principles right (and it almost certainly won’t be implemented as an actual wire in a gas cloud setup).