I had a conversation as a tangent to the previous open thread that left off with an unanswered question, so I’m reposting the question here.
It seems like the scheme I’ve been proposing here is not a common one. So how do people usually express the obvious difference between a probability estimate of 50% for a coin flip (unlikely to change with more evidence) vs. a probability estimate of 50% for AI being developed by 2050 (very likely to change with more evidence)?
That is precisely what I was proposing, just he explains it much better of course. Thanks!
In the subsequent article he makes essentially the same argument as Lumifer’s point about this having the potential to be turtles all the way down.
In a comment to the first article the author quotes Jaynes as saying:
What are we doing here? It seems almost as if we are talking about the ‘probability of a probability’. Pending a better understanding of what that means, let us adopt a cautious notation that will avoid giving possibly wrong impressions. We are not claiming that P(Ap|E) is a ‘real probability’ in the sense that we have been using that term; it is only a number which is to obey the mathematical rules of probability theory.
The “pending a better understanding of what that means” is also what I’ve been grappling with. In last week’s thread I initially proposed looking at it as the likelihood that I’ll find evidence that will make me change my probability estimate, and then I modified that to being how strong the evidence would have to be to make me change my probability estimate. [Aside from just understanding what “probabilities of probabilities” would actually mean, these ways of expressing it make the concept much more universally applicable than the narrow cases that the linked article is referring to.] But is there a better way of understanding it?
We can simplify this even further, to a fair coin versus an unknown weighted coin.
One way of viewing the difference is to say that you have different causal models of the two situations—with an unknown weighted coin there is an extra parameter to gather evidence about, therefore gathering evidence does more to your model of the world.
Well, your CI does change for the coin, if you observe strange artifacts of construction, or if the tosser has read Jaynes (who describes a way to cheat at coin tossing), or if the coin shows significant bias after lots of tries.
If you doubt this last bit, try a calibration app and look at one of your estimation buckets and ask yourself the same question: is my 70% bucket miscallibrated, or is this an effect of Tyche?
Your example constrains the evidence on the coin, by the convention that is attached to coin metaphors.
The less crappy response is that I like your attempt at illustrating the effects of prior distributions and general uncertainty, and that you should try another variation and see if it works better.
Sorry, not sure I got all that. As I mentioned in the previous conversation, I haven’t gotten to any of the more advanced stuff yet (or really anything past beginner level). Could you maybe try to rephrase that so I could understand better? Thanks.
I had a conversation as a tangent to the previous open thread that left off with an unanswered question, so I’m reposting the question here.
Your scheme seems to be Jaynes’s Ap distribution, discussed on LW here.
That is precisely what I was proposing, just he explains it much better of course. Thanks!
In the subsequent article he makes essentially the same argument as Lumifer’s point about this having the potential to be turtles all the way down.
In a comment to the first article the author quotes Jaynes as saying:
The “pending a better understanding of what that means” is also what I’ve been grappling with. In last week’s thread I initially proposed looking at it as the likelihood that I’ll find evidence that will make me change my probability estimate, and then I modified that to being how strong the evidence would have to be to make me change my probability estimate. [Aside from just understanding what “probabilities of probabilities” would actually mean, these ways of expressing it make the concept much more universally applicable than the narrow cases that the linked article is referring to.] But is there a better way of understanding it?
We can simplify this even further, to a fair coin versus an unknown weighted coin.
One way of viewing the difference is to say that you have different causal models of the two situations—with an unknown weighted coin there is an extra parameter to gather evidence about, therefore gathering evidence does more to your model of the world.
I don’t know if this is common, but perhaps you can use error bars on the probability estimates? So the coin is 50% +- 0.1%, but the AI is 50% +- 20%.
That’s essentially what I was proposing. See the linked article by one_forward above.
Well, your CI does change for the coin, if you observe strange artifacts of construction, or if the tosser has read Jaynes (who describes a way to cheat at coin tossing), or if the coin shows significant bias after lots of tries.
If you doubt this last bit, try a calibration app and look at one of your estimation buckets and ask yourself the same question: is my 70% bucket miscallibrated, or is this an effect of Tyche?
Your example constrains the evidence on the coin, by the convention that is attached to coin metaphors.
The less crappy response is that I like your attempt at illustrating the effects of prior distributions and general uncertainty, and that you should try another variation and see if it works better.
Retracted on the basis that I had not read the original thread and I almost certainly misunderstood the underlying question.
Sorry, not sure I got all that. As I mentioned in the previous conversation, I haven’t gotten to any of the more advanced stuff yet (or really anything past beginner level). Could you maybe try to rephrase that so I could understand better? Thanks.