Confidence level: strongly held, mostly opinionated, based on observation of (imo) bad LW norms.
We should stop using the phrase “epistemic status” and start using “confidence level.” In principle, “epistemic status” is meant to convey richer meta-information than confidence alone (i.e. the kind of evidence or how seriously a claim should be taken). In practice, it almost never does—on LW it’s usually just a clunkier way of saying “x confidence.”
If we actually want to convey more with less, we should just say “confidence level” and briefly qualify it with the relevant epistemic details (I.e. “low confidence, based on analogy,” or “high confidence, but mostly theoretical”). That’s clearer, less in-group-y, and lower friction. I think this is a good way to save up some weirdness points.
(Alternatively, one can used “qualified confidence”—a bit more jargony but traded for a bit more accuracy, though I perosonally like confidence level most).
Words have meanings. Confidence level is about my conclusion; epistemic status is about how I got there. These are different things, and the terms should be used accordingly.
I find epistemic status far more informative than confidence level. What can I do with someone’s “70%”? Nothing.
Epistemic Status: My best guess(but, epistemic effort was “talked to like 2-3 people about it and it felt good to each of us”) —Source
Epistemic status: Exploratory, speculative, half-baked thought —Source
Epistemic Status: I’ve really spent some time wrestling with this one. I am highly confident in most of what I say. However, this differs from section to section. I’ll put more specific epistemic statuses at the end of each section. —Source
Epistemic Status: Eliezer Yudkowsky writing the sequences. They sentenced me to twenty years of boredom. Galileo. This army. Chris Christie to Marco Rubio at the debate. OF COURSE! A woman scorned. For great justice. The Fire of a Thousand Suns. Expelling the moneylenders from the Temple. My Name is Susan Ivanova and/or Inigo Montoyo. You killed my father. Prepare to die. Indeed. It’s a trap. Tomfidence. I swear on my honor. End this. I know Kung Fu. Buckle up, Rupert. May the Gods strike me down to Bayes Hell. Compass Rose. A Lannister paying his debts. The line must be drawn here. This far, no farther. They may take our lives, but they will never take our freedom. Those who oppose me likely belong to the other political party. Ball don’t lie. Because someone has to, and no one else will. I’ve always wanted to slay a dragon. Persona! —Source
I generally don’t care much about people’s confidence levels. I don’t Aumann agree that hard. But I do care how much effort someone has put in, how settled an idea is, whether is has been helpful or predictive. “Epistemic status: personal experience” is directly useful to me. I’ll judge probability on merits however confident someone is (maybe not if I know their calibration curves, but I don’t), but if I know what effort they did and didn’t put in, I’ll happily directly update on that. I don’t think it’s factually true that epistemic status ‘almost never’ conveys something other than a confidence level.
Epistemic status: did a few minutes informal searching to sanity check my claims, which were otherwise off the cuff.
I’ve seen it used as “here’s how rigorous my methodology was”, and that seems meaningful. I can be very confident in something based on vibes alone[1], or I can run a series of intensive statistical tests and still be somewhat shaky about the apparent conclusion[2].
“I’ve trained a random forest classifier on several thousand amateur boxing matches, and got 95 percent accuracy on my validation set after making sure there was no cross contamination. The resulting model says that the fellow with the big glasses is likely to win a bout against the guy in the tank top, but I still wouldn’t put money on it.”
Thinking about this more. How about “epistemic humility note” or some such thing. Why that: I think saying “humility” is good self-prompting and reader-signaling. Seems similar purpose to “epistemic status” but leaves less room for accidentally dodging that purpose, though of course some room remains. Also similar but maybe stronger: maybe gives you more of a pass to be wrong in the reader’s mind (they don’t think you think you’re right). Also similar but maybe stronger: reminding people who are inclined to believe impressively collected results that those results can still be wrong, misleading, barking up the wrong tree, etc.
Epistemic humility note: This is just some stuff I think. I might be wrong. I’m not sure how I came to these opinions and I can’t prove them to you yet. Read if you believe I have an interesting track record of saying useful stuff, but don’t believe me if I assert something.
Epistemic humility note: This is strictly reporting of data I gathered via mechanical means, and detailed description of what those means were. Any attempt to come to a conclusion is delegated to the reader, who in turn might be reading the data wrong.
Epistemic humility note: I have none. No epistemic humility. I am right, and you should believe what I say.
Epistemic humility note: I’m pretty sure I gathered the right data and have interpreted it correctly, but if you don’t agree, please tear me to shreds.
Epistemic humility note: Peer reviewed paper. We checked. We’re sure we’re right on what we report.
that last one reads as a warning sign to me, but seems easier to write than “Epistemic status: {same sentence}”.
Confidence level: strongly held, mostly opinionated, based on observation of (imo) bad LW norms.
We should stop using the phrase “epistemic status” and start using “confidence level.” In principle, “epistemic status” is meant to convey richer meta-information than confidence alone (i.e. the kind of evidence or how seriously a claim should be taken). In practice, it almost never does—on LW it’s usually just a clunkier way of saying “x confidence.”
If we actually want to convey more with less, we should just say “confidence level” and briefly qualify it with the relevant epistemic details (I.e. “low confidence, based on analogy,” or “high confidence, but mostly theoretical”). That’s clearer, less in-group-y, and lower friction. I think this is a good way to save up some weirdness points.
(Alternatively, one can used “qualified confidence”—a bit more jargony but traded for a bit more accuracy, though I perosonally like confidence level most).
Words have meanings. Confidence level is about my conclusion; epistemic status is about how I got there. These are different things, and the terms should be used accordingly.
I find epistemic status far more informative than confidence level. What can I do with someone’s “70%”? Nothing.
A bunch of examples of it being used:
And of course, my favorite of all time:
Posts come with poems
about how you think you know
what you think you know.
Could someone point to an example of “epistemic status” used correctly, where you couldn’t just substitute it with “confidence level”?
I generally don’t care much about people’s confidence levels. I don’t Aumann agree that hard. But I do care how much effort someone has put in, how settled an idea is, whether is has been helpful or predictive. “Epistemic status: personal experience” is directly useful to me. I’ll judge probability on merits however confident someone is (maybe not if I know their calibration curves, but I don’t), but if I know what effort they did and didn’t put in, I’ll happily directly update on that. I don’t think it’s factually true that epistemic status ‘almost never’ conveys something other than a confidence level.
Epistemic status: did a few minutes informal searching to sanity check my claims, which were otherwise off the cuff.
Yeah but I don’t think OP meant that by using “confidence level” you have to give a percentage. You can just swap out the phrase. Your two examples:
Confidence level: personal experience
Confidence level: did a few minutes informal searching to sanity check my claims, which were otherwise off the cuff.
I think these still work perfectly well, and now they are understandable to a much larger set of people.
The first does not work at all. “Personal experience” is not a level of confidence, and does not imply any particular level of confidence.
I’ve seen it used as “here’s how rigorous my methodology was”, and that seems meaningful. I can be very confident in something based on vibes alone[1], or I can run a series of intensive statistical tests and still be somewhat shaky about the apparent conclusion[2].
“That guy in the black T-shirt looks mean, I bet he could win a fight against that guy in the leather jacket.”
“I’ve trained a random forest classifier on several thousand amateur boxing matches, and got 95 percent accuracy on my validation set after making sure there was no cross contamination. The resulting model says that the fellow with the big glasses is likely to win a bout against the guy in the tank top, but I still wouldn’t put money on it.”
Thinking about this more. How about “epistemic humility note” or some such thing. Why that: I think saying “humility” is good self-prompting and reader-signaling. Seems similar purpose to “epistemic status” but leaves less room for accidentally dodging that purpose, though of course some room remains. Also similar but maybe stronger: maybe gives you more of a pass to be wrong in the reader’s mind (they don’t think you think you’re right). Also similar but maybe stronger: reminding people who are inclined to believe impressively collected results that those results can still be wrong, misleading, barking up the wrong tree, etc.
that last one reads as a warning sign to me, but seems easier to write than “Epistemic status: {same sentence}”.
This post is very high epistemic status. Please show it the epistemic respect it epistemicologically deserves.