I guess the problem is with the “knows more” words. It’s not just how many bits of information you get, but also how are they related to your question. As a trivial example, it would be better to have 1 relevant bit of information than 1024 bits of irrelevant information. In this example, all information is relevant, but differently.
Imagine the following situation: You have letters “A”, “B”, “C”, “D” and you randomly choose one of them.
You have two participants in the experiment. To the first participant you tell that you did not choose “A”. To the second participant you tell that you did not choose “B”. Each of them has the same amount of information, right?
Then you ask them whether the letter you chose was a consonant. The first one says “Certainly yes.” The second one says “I am not sure, but with probability 66% yes.”
How is it possible that the same amount of information gives them different certainty? The answer is, the same amount of information in general is not necessarily the same amount of information about the question you gave them.
I guess the problem is with the “knows more” words. It’s not just how many bits of information you get, but also how are they related to your question. As a trivial example, it would be better to have 1 relevant bit of information than 1024 bits of irrelevant information. In this example, all information is relevant, but differently.
Imagine the following situation: You have letters “A”, “B”, “C”, “D” and you randomly choose one of them.
You have two participants in the experiment. To the first participant you tell that you did not choose “A”. To the second participant you tell that you did not choose “B”. Each of them has the same amount of information, right?
Then you ask them whether the letter you chose was a consonant. The first one says “Certainly yes.” The second one says “I am not sure, but with probability 66% yes.”
How is it possible that the same amount of information gives them different certainty? The answer is, the same amount of information in general is not necessarily the same amount of information about the question you gave them.