Alice: Are there any examples of X?
Bob: Here.
Alice: Okay, but are there any examples of X’?
Bob: Yes, here.
Alice: That has feature Y, which-
Bob: Dude, stop moving the goalposts.
The thing is, I think in this situation, Alice often isn’t moving the goalposts. Instead, she’s looking for something, and not fully specifying it up-front. Maybe because she’s speaking imprecisely. Maybe because she doesn’t know quite what she’s looking for, but does know when she sees something that that’s not it.
Alice: Any research showing that supplement helps?
Bob: Yes, here’s a trial.
Alice: Anything not funded by the manufacturer?
Bob: This one was funded by the NIH.
Alice: Thanks, but it’s n=12 and there’s no control arm.
Bob: ಠ_ಠ
What Alice wants to know here is should she believe the supplement helps. Asking for research is a proxy for that, and the research Bob finds answers her proxy but doesn’t say much about the real question.
This is probably frustrating for Bob! If he doesn’t know why Alice is asking, or doesn’t know why the things he finds don’t help her, it might look to him a lot like she’s moving the goalposts / fighting a rearguard retreat against the evidence. Like she’s just looking for reasons to dismiss what he’s bringing up, so she doesn’t have to change her mind. But it’s an importantly different thing going on.
If Bob thinks this kind of thing might be going on, he might ask: “okay, help me understand here what you’re actually looking for? Right now I expect that if I give another example, it’ll be wrong again in some way I can’t predict.”
And if Alice notices it, she might say: “sorry, I know what this looks like—the reason I’m asking is…”.
Alternate possible framings:
Alice is playing the gotcha game. She believes !X and thinks that every failed/rebutted evidence increases her status or bolsters her position. See also “Isolated Demand for Rigor”
Bob is a literalist who is trying to answer her direct question, without wondering (or asking) what is the context and purpose of this exploration.
In any of these cases, using more words helps a lot. Alice should give context and more description of what she’s uncertain about, and Bob should engage more deeply in the overall debate/discussion rather than trying to just provide the narrow thing Alice’s words describe.
Alice needs to work on her context engineering skills. Bob needs to practice more active listening and broad awareness. The problem with asking an expert a question is that they will try to answer it. Most people won’t notice the question behind the question, look for the thing they don’t know how to ask, and make the implicit explicit.
Yep. My personal definition of AGI unapologetically amounts to “I will know it when I see it”.
I can ask someone: ‘How many things are there in the world?‘. They can say ‘Are two different numbers the same thing?’ or ‘Are all apples the same thing or is each instantiation of an apple a separate thing?’.
These rules are not agreed on before hand and so the participants must try to map their internal representations to the other person’s.
In your example of the supplement trial, Alice assumes some level of agreement on what it would mean for what reasonable research is. And only after each step in discussion are Alice’s tacit assumptions revealed or unshrouded by a Wittgensteinian ‘Language Game’.
This is a classical Lakatosian conflict. In his terms, Alice is doing something called ‘monster-barring’: Alice has an (implicit) ‘conjecture’, Bob produces (counter)examples which Alice bars by adding conditions on what counts as an example. Your defense is also in Lakatos: the counterexamples which at first are barred are eventually synthesized into the ‘proof-generated concept’ which is the definition Alice did not have when she started (or the conjecture is bled into vacuity, which is informative either way). A more productive form of discourse is lemma-incorporation (identify what the counterexample actually refutes and write it as a condition to be satisfied).
There is also an answer there for how one detects whether or not the goalposts have actually shifted or if there really is a refinement towards a correct domain of validity: one observes whether or not the successive barrings decrease content without adding anything back. Since in her case the conjecture is a negative one (there does not exist an X), the content must come from the evidential standard (have you excluded every achievable way to prove X, such that you can no longer prove anything); is the standard applicable globally or is there a reason why it is applied locally; does the exclusion criteria kill only the specific study that was brought up or does it produce a rule for looking for what studies might count somewhere, etc.
Since the question and part of the answer match very closely, I think it would be sensible for you to read on this perspective (not a hard philosopher to read, nor uncommon; the thing to look at is Proofs & Refutations, which is a dialogue on V—E + F = 2, easy to digest for anyone with the typical backgrounds here)