I don’t know how shminux interprets the words, and if the question was related to this, but there is an issue in your use of “impossible”. Things that happen in this world are actual, and things that happen in alternative worlds are possible. (The set of alternative worlds needs to be defined for each question.) An impossible situation is one that can’t occur, that doesn’t happen in any of the alternative worlds.
Thus outputs of an algorithm different from the actual output are not actual, and furthermore not possible, as they don’t occur in the alternative worlds with the same algorithm receiving the same inputs. But there are alternative worlds (whose set is different than in the previous sentence, no longer constrained by the state of input) with the same algorithm that receives different inputs, and so despite not being actual, the different inputs are still possible, in other words not impossible.
I agree with everything you said, it seems like common sense. I’m going by the quote above, and by the similar sentiment in multiple MIRI papers, that there are “logically impossible worlds” described by the counterfactuals like “same agent, same input, different outcome”. I am lost as to why this terminology is useful at all. Again, odds are, I am missing something here.
An input to another algorithm may be our source code, with the other algorithm’s output depending on what they can prove about our output. If we assume they reason consistently, and want to prove something about their output, we might assume what they prove about us even when that later turns out impossible.
I don’t know how shminux interprets the words, and if the question was related to this, but there is an issue in your use of “impossible”. Things that happen in this world are actual, and things that happen in alternative worlds are possible. (The set of alternative worlds needs to be defined for each question.) An impossible situation is one that can’t occur, that doesn’t happen in any of the alternative worlds.
Thus outputs of an algorithm different from the actual output are not actual, and furthermore not possible, as they don’t occur in the alternative worlds with the same algorithm receiving the same inputs. But there are alternative worlds (whose set is different than in the previous sentence, no longer constrained by the state of input) with the same algorithm that receives different inputs, and so despite not being actual, the different inputs are still possible, in other words not impossible.
I agree with everything you said, it seems like common sense. I’m going by the quote above, and by the similar sentiment in multiple MIRI papers, that there are “logically impossible worlds” described by the counterfactuals like “same agent, same input, different outcome”. I am lost as to why this terminology is useful at all. Again, odds are, I am missing something here.
An input to another algorithm may be our source code, with the other algorithm’s output depending on what they can prove about our output. If we assume they reason consistently, and want to prove something about their output, we might assume what they prove about us even when that later turns out impossible.