“Intent to inform” jives with my sense of it much more than “tell the truth.”
On reflection, I think the ‘epistemic peer’ thing is close but not entirely right. Definitely if I think Bob “can’t handle the truth” about climate change, and so I only talk about AI with Bob, then I’m deciding that Bob isn’t an epistemic peer. But if I have only a short conversation with Bob, then there’s a Gricean implication point that saying X implicitly means I thought it was more relevant to say than Y, or is complete, or so on, and so there are whole topics that might be undiscussed because I don’t want to send the implicit message that my short thoughts on the matter are complete enough to reconstruct my position or that this topic is more relevant than other topics.
---
More broadly, I note that I often see “the discourse” used as a term of derision, I think because it is (currently) something more like a marketing war than an open exchange of information. Or, like a market left to its own devices, it has Goodharted on marketing. It is unclear to me whether it’s better to abandon it (like, for example, not caring about what people think on Twitter) or attempt to recapture it (by pushing for the sorts of ‘public goods’ and savvy customers that cause markets to Goodhart less on marketing).
A class I took in graduate school worked this way; here’s the professor’s paper about it. Some notes on how it worked:
He used the logarithmic scoring rule, and normalized it such that a maxent guess was 0 points.
It takes students a while to learn calibration, and so it’s worth doing many small-stakes versions of this before doing large-stakes versions of it. (The way he did this—one question as a homework assignment each week, and then one or two large exams—didn’t do all that well for this, especially since the homework assignments didn’t fully replicate the “how well can I interpret the question without asking for clarification?” part of the uncertainty that was relevant on tests.)
Getting probabilities from the students lets you generate average probabilities for each answer, which is actually quite useful at figuring out where the class is confused. Importantly, you can tell the difference between a question where the average estimate on the right answer is 90% and one where the average estimate on the right answer is 50%, even though both of those will look almost identical in the world where students only choose their top answer!