We could try something similar even without Lojban, just in written English. We could include the ‘evidentials’ in text using some agreed upon notation. I guess it would be inconvenient at first, but later it could become more easy.
I just cannot imagine that I would seriously estimate the probability of every sentence I write. That would make my writing too slow, or I would just assign some arbitrary probabilities—like 80% or 95% -- to most things.
Evidentials in Lojban are optional, not mandatory; and most of the point of this exercise, for me, is to see if I can improve the probability I assign to sentences from being simply arbitrary. I know that as I started this, many of the probabilities I assigned have been laughably inaccurate—and I haven’t been able to think of a way to improve my estimation any faster than by constantly practicing my estimations.
Using probabilities is assuming too much information you don’t actually know. Probabilities depends on calibration, good math, understanding the massive difference between 98%+1% and 50%+1%, and so on.
I’d use more qualitative measures that could be mapped to quantities if need be. Tautology, strong disbelief, disbelief, weak disbelief, etc.
understanding the massive difference between 98%+1% and 50%+1%
This is one of the more valuable lessons that using logarithmic decibels, instead of linear probabilities, provides. Going from 98% to 99% adds 3 decibels; going from 50% to 51% adds 0.17 decibels.
Qualitative measures are fine—Lojban even has a ‘number’ word meaning ‘about’ (“ji’i”), and even getting a rough feel for the confidence-levels involved can be a step up from not having any idea at all, and is a step closer to having a better calibration for quantitative measures.
Well, here’s a first stab. We only need to cover 50-100% since English gives us negations: “unlikely” versus “likely”, “unprobable” versus “probable”. (If we can express 60% and we want to express 40%, we can just negate whatever we say for 60%.) Going by the above scheme, the 50-100% range requires 13 modifiers. If I replace the >99% for 99%, which I don’t think is very useful, I need 13 or so. For infinity or 100%, I think it’s better to signal a discontinuity by using a pair like “certain”/”impossible” (neg infinity or 0%) - since they aren’t probabilities. I originally set had the range start at 50%, but then it was pointed out that negation was funny, so I realize I had to make that a special word as well, along with 0% and 100%. Half-way through, if I switch from “likely” to “probable” I can reuse the previous modifiers, so I only need to think of 5-6 modifiers.
It turns out that this is a really hard balancing act. I think I’m roughly satisfied with:
I’ll think about this for a while more, and then I think I’ll go through gwern.net and try to rationalize all uses of informal probability to use this scheme. I’m calibrated, so I might as well get some mileage out of it!
Translating numerical probabilities into verbal labels has been an active area of research. As an entry point into that literature, see the review article Teigen & Brun (2003, Verbal expressions of uncertainty and probability).
You might want to take a look at some of the other attempts out there to try to come up with labels that are more intuitive (I see “likely” and “probable” as equivalent, which would make this system where “somewhat probable” > “extremely likely” very unintuitive for me). Teigen and Brun cite several attempts which “have been made to construct standard lists of verbal expressions, where each phrase is coordinated with an appropriate numeric probability (Beyth-Marom, 1982; Hamm, 1991; Tavana, Kennedy & Mohebbi, 1997; Renooij & Witteman, 1999).” The full citations for those 4 papers are:
Beyth-Marom, R.(1982). How probable is probable? A numerical translation of verbal probability expressions. Journal of Forecasting, 1, 257–269. Hamm, RM (1991). Selection of verbal probabilities: A solution for some problems of verbal probability expression. Organizational Behavior and Human Decision Processes, 48, 193–223. Tavana, M., Kennedy, DT & Mohebbi, B.(1997). An applied study using the analytic hierarchy process to translate common verbal phrases to numerical probabilities. Journal of Behavioral Decision Making, 10, 133–150. Renooij, S. & Witteman, C.(1999). Talking probabilities: Communicating probabilistic information with words and numbers. International Journal of Approximate Reasoning, 22, 169–194.
After reading through those cited papers, I think the Kessler scale is still the best of the suggestions and simpler than my own suggestion. I guess I’ll just use that in the future. I’ve made some flashcards to help me memorize them.
Kesselman’s thesis suggests this mapping: Kesselman List of Estimative Words
100% Certainty
86-99% Highly Likely
56-70% Likely
46-55% Chances a Little Better [or Less]
31-45% Unlikely
13-30% Highly Unlikely
1-15% Remote
0% Impossibility
I find the middle phrasing entirely unsatisfactory (“possible” is an obvious replacement), and the chunking is a little crude, but I do agree it should be impossible for most people to get the relative rankings wrong and invert any pairs. Not sure if it’s better or not; need to read some of your cites, although the review’s various PDF homes are all dead right now. EDIT: the book is available though.)
It seems to me that the way we talk about possibilities in English has a component of downside risk that isn’t well captured by that ranking, especially at the extremes. I might comfortably refer to a 15% chance of losing twenty dollars as “remote”, but the same certainly wouldn’t be true of a 15% chance of losing my life.
I find the middle phrasing entirely unsatisfactory (“possible” is an obvious replacement)
“Possible” seems to have two distinct meanings. The first one fits your usage, but the other is more of a binary expression, used to express the fact that something is not impossible. In other words, anything whose probability is equal or greater than 1% (say) can be tagged with “possible”, and using this sense of “possible” for the 46-55% range seems wrong—it would deserve a stronger word. To avoid the risk of confusion about which sense is meant, I suggest using something like “entirely possible”.
The recurrent findings in these studies are (1) a reasonable degree of between-group consistency, combined with (2) a high degree of within-group variability. In other words, mean estimates of “very probable”, “doubtful” and “improbable” are reasonably similar from study to study, supporting the claim that probability words are translatable; but, at the same time, the interindividual variability of estimates is large enough to represent a potential communication problem. If, for instance, the doctor tells the patient that a cure is “possible”, she may mean a 5 per cent chance, but it may be interpreted to mean a 70 per cent chance, or vice versa. This variability is typically underestimated by the participants themselves. Brun and Teigen (1988) asked medical doctors to specify a range within which would fall 90 per cent of other doctors’ interpretations. This interval included on the average (for 14 verbal phrases) less than 65 per cent of the actual individual estimates. Amer, Hackenbrack and Nelson (1994) found that auditors’ 90 per cent ranges included, on average, only 56 per cent of the individual estimates (for 23 phrases). In other words, the problem posed by interindividual variability appears to be aggravated by a low degree of variability awareness.
...several attempts have been made to construct standard lists of verbal expressions, where each phrase is coordinated with an appropriate numeric probability (Beyth-Marom, 1982; Hamm, 1991; Tavana, Kennedy & Mohebbi, 1997; Renooij & Witteman, 1999)
...Verbal phrases are, furthermore, parts of ordinary language, and thus sensitive to conversational implicatures. So I may say that a particular outcome is somewhat uncertain, not because I think it has a low probability of occurring, but because I want to modify some actual, imagined or implied belief in its occurrence. Such modifications can go in two directions, either upwards or downwards on the probability scale. Verbal probability expressions can accordingly be categorised as having a positive or a negative directionality. They determine whether attention should be directed to the attainment or the non-attainment of the target outcome, and, in doing so, they have the ability to influence people’s judg- ments and decisions in an unambiguous way. Words may be denotatively vague, but they are argumentatively precise. If you tell me that success is “possible”, I know I am being encouraged, even if I do not know whether you have a probability of 30 per cent or of 70 per cent in mind. If you say it is “not certain”, I know I am advised to be careful and to think twice. But if you tell me there is a 45 per cent probability I will not know what to think. The information is precise, but its pragmatic meaning is undecided. Do you mean uncertainty (I have only a 45 per cent chance) or possibility (at least I have a 45 per cent chance)? Likelihood or doubt? Or both?
We could try something similar even without Lojban, just in written English. We could include the ‘evidentials’ in text using some agreed upon notation. I guess it would be inconvenient at first, but later it could become more easy.
I just cannot imagine that I would seriously estimate the probability of every sentence I write. That would make my writing too slow, or I would just assign some arbitrary probabilities—like 80% or 95% -- to most things.
Evidentials in Lojban are optional, not mandatory; and most of the point of this exercise, for me, is to see if I can improve the probability I assign to sentences from being simply arbitrary. I know that as I started this, many of the probabilities I assigned have been laughably inaccurate—and I haven’t been able to think of a way to improve my estimation any faster than by constantly practicing my estimations.
Using probabilities is assuming too much information you don’t actually know. Probabilities depends on calibration, good math, understanding the massive difference between 98%+1% and 50%+1%, and so on.
I’d use more qualitative measures that could be mapped to quantities if need be. Tautology, strong disbelief, disbelief, weak disbelief, etc.
This is one of the more valuable lessons that using logarithmic decibels, instead of linear probabilities, provides. Going from 98% to 99% adds 3 decibels; going from 50% to 51% adds 0.17 decibels.
Qualitative measures are fine—Lojban even has a ‘number’ word meaning ‘about’ (“ji’i”), and even getting a rough feel for the confidence-levels involved can be a step up from not having any idea at all, and is a step closer to having a better calibration for quantitative measures.
Well, here’s a first stab. We only need to cover 50-100% since English gives us negations: “unlikely” versus “likely”, “unprobable” versus “probable”. (If we can express 60% and we want to express 40%, we can just negate whatever we say for 60%.) Going by the above scheme, the 50-100% range requires 13 modifiers. If I replace the >99% for 99%, which I don’t think is very useful, I need 13 or so. For infinity or 100%, I think it’s better to signal a discontinuity by using a pair like “certain”/”impossible” (neg infinity or 0%) - since they aren’t probabilities. I originally set had the range start at 50%, but then it was pointed out that negation was funny, so I realize I had to make that a special word as well, along with 0% and 100%. Half-way through, if I switch from “likely” to “probable” I can reuse the previous modifiers, so I only need to think of 5-6 modifiers.
It turns out that this is a really hard balancing act. I think I’m roughly satisfied with:
I’ll think about this for a while more, and then I think I’ll go through gwern.net and try to rationalize all uses of informal probability to use this scheme. I’m calibrated, so I might as well get some mileage out of it!
Translating numerical probabilities into verbal labels has been an active area of research. As an entry point into that literature, see the review article Teigen & Brun (2003, Verbal expressions of uncertainty and probability).
You might want to take a look at some of the other attempts out there to try to come up with labels that are more intuitive (I see “likely” and “probable” as equivalent, which would make this system where “somewhat probable” > “extremely likely” very unintuitive for me). Teigen and Brun cite several attempts which “have been made to construct standard lists of verbal expressions, where each phrase is coordinated with an appropriate numeric probability (Beyth-Marom, 1982; Hamm, 1991; Tavana, Kennedy & Mohebbi, 1997; Renooij & Witteman, 1999).” The full citations for those 4 papers are:
Beyth-Marom, R.(1982). How probable is probable? A numerical translation of verbal probability expressions. Journal of Forecasting, 1, 257–269.
Hamm, RM (1991). Selection of verbal probabilities: A solution for some problems of verbal probability expression. Organizational Behavior and Human Decision Processes, 48, 193–223.
Tavana, M., Kennedy, DT & Mohebbi, B.(1997). An applied study using the analytic hierarchy process to translate common verbal phrases to numerical probabilities. Journal of Behavioral Decision Making, 10, 133–150.
Renooij, S. & Witteman, C.(1999). Talking probabilities: Communicating probabilistic information with words and numbers. International Journal of Approximate Reasoning, 22, 169–194.
After reading through those cited papers, I think the Kessler scale is still the best of the suggestions and simpler than my own suggestion. I guess I’ll just use that in the future. I’ve made some flashcards to help me memorize them.
Kesselman’s thesis suggests this mapping: Kesselman List of Estimative Words
100% Certainty
86-99% Highly Likely
56-70% Likely
46-55% Chances a Little Better [or Less]
31-45% Unlikely
13-30% Highly Unlikely
1-15% Remote
0% Impossibility
I find the middle phrasing entirely unsatisfactory (“possible” is an obvious replacement), and the chunking is a little crude, but I do agree it should be impossible for most people to get the relative rankings wrong and invert any pairs. Not sure if it’s better or not; need to read some of your cites, although the review’s various PDF homes are all dead right now. EDIT: the book is available though.)
It seems to me that the way we talk about possibilities in English has a component of downside risk that isn’t well captured by that ranking, especially at the extremes. I might comfortably refer to a 15% chance of losing twenty dollars as “remote”, but the same certainly wouldn’t be true of a 15% chance of losing my life.
“Almost Certain” is missing and “Highly Likely” and “Higly Unlikely” have the wrong numbers. It should be:
100% Certainty
86-99% Almost Certain
71-85& Highly Likely
56-70% Likely
46-55% Chances a Little Better [or Less]
31-45% Unlikely
16-30% Highly Unlikely
1-15% Remote
0% Impossibility
“Possible” seems to have two distinct meanings. The first one fits your usage, but the other is more of a binary expression, used to express the fact that something is not impossible. In other words, anything whose probability is equal or greater than 1% (say) can be tagged with “possible”, and using this sense of “possible” for the 46-55% range seems wrong—it would deserve a stronger word. To avoid the risk of confusion about which sense is meant, I suggest using something like “entirely possible”.
To me, ‘entirely possible’ doesn’t convey around 50-50; so why bother sticking in an entire other word?
Notes from Teigen & Brun:
The cached HTML of the review is available.