Experimental evidence of the value of redundant oral tradition
Something I’ve been hearing a lot lately (specifically from Orthodox Jews, although it comes up a lot in debates about religion) is that having a large number of people telling a story makes it more likely the story is true, because multiple witnesses can call each other out for deviating from the truth.
My gut reaction is that this is extremely false. But it’s a point that should be scientifically testable, and I figure that someone should have done a study on it by now. Does anyone know of such a thing?
A related issue is the argument that oral tradition meant something very different thousands of years ago, when it was the ONLY form of historical record. Oral historians were duty-bound to preserve the story. This sounds plausible. It probably ISN’T as easily testable since we can’t compare oral history from pre-writing times against… well, much of anything. (Well, I guess archaeological evidence, if the events being described would have left enough archaeological evidence). Is there an official, accepted scholarly opinion on this?
The argument as used by Orthodox Jews is an interesting case study of what I think of as “unique apologetics.” There’s a large body of apologetics that is structurally very similar across a wide swatch of religions. However, many religions have small bits that give their apologetics a unique flavor, a way of at least superficially distinguishing their religion from others. For example, among Islamic apologetics, one sees the argument that a founder of a religion that was false would likely claim to be divine, and that Muhammad’s not doing so is evidence of his sincerity/the truth of the religion.
Among Orthodox Jews, the argument of a large scale, simultaneous oral tradition dates back to the Kuzari, an apologetic series of dialogues by Yehuda Halevi from the 1100s. The argument in its most general form does sound valid: if one did have a few million lines of separate transmission this would be quite strong evidence. The problem isn’t really in the lines of the argument but in the factual premises of the details. There are some aspects where the ancient Israelite oral tradition seems remarkably accurate (see for example, the genetic research about kohanim). However, there are not a million separate lines of transmission but rather a large number of people interacting and talking to each other. Now, arguably that should reduce the total amount of memetic drift (I’m possibly stretching the meme-gene analogy too far here but the idea should be clear). But the oral tradition itself is shown to be severely mutable in many respects. One amusing example of this failing is a line in the Talmud that discusses the importance of properly attributing ideas and sayings to their actual authors. An identical line shows up in multiple locations, each time attributed to a different Rabbi. This also helps answer your question and suggests that oral history can be frequently extremely unreliable, even in cases where the people in question feel a major moral/ethical duty to pass it on reliably.
Moreover, the combination of both written and oral tradition itself testifies strongly to the tradition having been repeatedly reduced to a bottleneck where at most a few people were aware of the tradition. For example, see 2 Chronicles Chapter 34 and 2 Kings Chapter 22 where under the time of King Josiah it is implied that almost everyone has completely forgotten much of the tradition until a copy of something (possibly a Torah, possibly the text of Deuteronomy) is discovered in a wall. This is not the only example of this sort of event recorded in the Bible. The Bible describes a similar set of activity at the time of Ezra.
So, the upshot is that in general it seems that oral history is not very reliable although there are a handful of striking examples of apparently accurate oral history. It isn’t clear how one would tell how frequent such incidents are. I wouldn’t rely on the Missionaria Protectiva.
And this is why I spent 20 years being Orthodox, having being trapped in High School...
And this is why I cut my losses..
All that due to a poor knowldege of the Scriptures :).
Sociologists have tested the degree to which myths and folktales in present day oral cultures vary with retellings and transmission. I’m not finding the study with a cursory search, but I recall that they found that although the tellers thought they were telling the same story each time, the stories actually changed, not just with transmission from one teller to another, but with retellings by the same trained storytellers, and sometimes in significant details. The sociologists had to pretty much dispense with the notion that highly trained individuals in an oral culture were able to effectively preserve the content of the stories passed down by word of mouth much better than individuals from cultures used to using the written word would be able to.
I’d really like to find that study. That seems pretty directly relevant.
It seems to be a bit more complicated than that. Still, even a Jesuit isn’t willing to trust it as history for more than 150 years.
I’m curious what the Orthodox Jews think about this more-recent instance of Mass Revelation. Multiple witnesses, but I suspect there wasn’t a lot of “calling each other out for deviating from the truth”.
There is no reason that even forms an argument. It just says the meme in question spreads well with fidelity—that it’s a good story, i.e. one that hooks into cognitive biases in a memorable fashion—and nothing about the truth or falsity of its informational content.
This reminds me of how conspiracy theories spread. I have looked into Project Blue Beam, for example. There is a LOT of material on the web about Project Blue Beam—lots of videos, web pages and so on about it. It’s quite popular on the web as conspiracy theories go. It all comes down to one short book and a talk by one person. But it captured the imagination of conspiracy theorists, and started accreting other conspiracy theories to it.
(The actual origin of Project Blue Beam—where that one person got it from—is hilarious, but I won’t spoil it for you.)
More people confirming a story is certainly epsilon more evidence that the story is correct (Because more people confirming a story being evidence that it is false is absurd).
A more interesting question is, what is the magnitude of epsilon in a case like the one described here? This is in principle testable, but I certainly don’t know exactly how to go about testing it.
Intuitively, I’d say it’s some sort of logarithm or quadratic curve—if one person tells me they say a black dog the next street over, that bumps up my belief a lot; if two people tell me it, it still increases, but not nearly as much; and so on to the point where if 2 billion people tell me that, I begin to think this is part of some cult and start lowering credence.
A better way to correct for errors in transmission of a body of knowledge (including stories) is to write the knowledge down (i.e. encode in a “dumb”, semi-permanent physical substrate), rather than rely on oral re-tellers to identify each others’ deviation from the average re-telling.
Well, duh.
But that’s not relevant to the current discussion, as the people in question had not yet invented the technology to do so.
Yes, but if the question is whether redundant oral re-tellers is an improvement (in transmission fidelity) over single re-tellers, the answer is trivial. And so is the comparison to writing.
So what’s the actual hard question here?
That’s not quite accurate in the original context. The Hebrew Bible was written well before the Talmud. The Talmud and associated texts claim to be oral traditions that were not written down when the Biblical text was written down. There’s a fair bit of evidence that at least some of the Talmudic texts did descend from old oral traditions.
A large # of people “telling a story” has very little intrinsic value (however, this is not the Kuzari argument).
A large # of people claiming to be eyewitnesses does have significant value (also not the Kuzari argument).
A chain of a large # of eyewitnesses (as in the case of the Kuzari argument), falls somewhere in between.
However, there are many other (far more) crucial factors such as:
The nature of the claim (e.g. plausibility, self-serving, compatibility with other information)
The amount of time & conditions (e.g. peaceful/tumultuous) thru which the tradition has passed.
The number of links in the chain
The quality/reliability of the links in the chain
In the case of the Kuzari argument (which is the underlying background the post), we’re dealing with:
a supernatural claim;
a narrative that does not fit with other known information (not just in the exodus narrative, but the biblical narrative in general);
a huge gap of time (~1,000 years from the claimed Sinai Theophany c. 1300 BC to the 2nd temple era, which included 2 exiles and returns) during which, not only is there no evidence of a widespread belief in the Sinai theophany, there is evidence against the notion, not only archeologically, but even from the biblical narrative itself. This leaves plenty of time for some event that was:
a. natural (e.g. earthquake, volcano, etc.), and/or
b. experienced by a small number of people
to evolve into a mass theophany.
This paper might be relevant—Kruskal’s “Miracles and Statistics: The Casual Assumption of Independence”.
Funny, my gut reaction is that this is extremely true.
If 1 or a million people tell me some thing, there are exactly 3 options available to me:
increase my belief in that thing by some amount
leave my belief unchanged
decrease my belief
Suppose my 50 neighbors—all nice normal folks—tell me in passing that they saw my brown dog loose in the morning. Why would I decrease my belief? Wouldn’t that be incredibly perverse?
And why would I ignore them and leave my belief unchanged? I don’t discriminate against any other information source. If my eyes tell me 50 times that my brown dog is loose, I don’t go ‘phsaw, who cares what you guys think’. I increase my belief that my dog is loose.
About any proposition in general, the more people tell you it, the more you believe it. Let’s generalize this even more: the more data you have telling you to believe some proposition, the more you believe that proposition. (What a shocking conclusion.)
Now, sometimes there are conspiracies, but they are extremely rare and unlikely and require involved explanations (like explaining the existence of religious belief in general does). Those Jews are right that the more testimony the greater the likelihood, but this truth does not prove the things they want it to prove.
tl;dr: your post is ill-founded and you need to explain why the Bayesian solution to the raven paradox is wrong.
Which is very poor practice if they were all convinced by each other.
Tell me, of the millions of things that you have been told, have more than half been about things where they were all convinced by each other?
Second question: do you not understand the relevance of the raven paradox? Epsilon evidence is still non-zero evidence. Bite that tiny bullet!
There’s still a chance, right?
No. That link is about a highly specific tiny statement with tons of more germane evidence to take into account, which drowns the epsilon evidence. I am not talking about any specific point.
Likewise, the raven paradox has tons of more germane evidence which outweighs the logical point of observing a non-raven if you deliberately miss the point.
I feel like I’m trying to discuss the trolley paradox, and everyone is going, ‘no, toss over your backpack instead of pushing the fat man or letting them die! Isn’t it obvious there are tons of other things you could do?’
(Someone help me out here, we must have an article about how people love to finesse logical or philosophical arguments in this sort of way. Oh wait, I guess we do: The Least Convenient Possible World.)
A million people is certainly more persuasive than one. But when we’re talking about a claim that has no other evidence supporting it, a far greater degree of incredulity is warranted. (By contrast, “I saw an brown dog” has LOADS of evidence supporting it to begin with, such as the fact that you’ve seen hundreds of dogs so you know they exist and are common, and that people see them from time to time).
More to the point, in this case, we don’t have a million people saying “I saw God.” We have millions of people saying “our parents said their parents said their parents said their parents heard from their parents that a while ago 1.5 million people saw God.” When you have millions of people sharing and retelling a story over the course of generations, I believe it is more likely that story gets distorted over time rather than remains constant. This was my point, not that I should ignore 1.5 people who claim to have witnessed an event.
Gwern commented because you said something technically wrong, even though it was essentially right. You denied that “having a large number of people telling a story makes it more likely the story is true”. But having a large number tell the story does make it more likely to be true. After all, conditioning on the story’s being false makes it less likely that a large number would be telling it (out of all possible false stories).
But your primary point remains: Often, even after conditioning on a story’s being told by millions, the probability of the story’s being true remains vanishingly small.
Which people are being referred to in your first paragraph? The original people writing down the books? Because there are contradictions in what was passed down.
Or are they counting just anyone who believes what they believe? Because then we have to get into independent evidence vs. non-independent evidence.
The arguments I’m dealing with are fundamentally flawed in more ways that I feel like recounting. There was just one particular area where I wasn’t actually sure what the data said and I wanted to find out for my own purposes.
The debate was about Mass Revelation. The story is that approximately 1.5 million Jews saw God, and told their children that they saw it, who told their children that their grandparents saw it, etc. 800 years of oral tradition pass before the story is written down.
I’m really tired right now and don’t feel like spelling out the argument in detail (it’s flawed in plenty of obvious ways that don’t warrant your valuable time). But the basic idea is that it’s impossible to convince millions of people that they saw something that they didn’t see, and if you tried to sell the story to a younger generation, the older generation would say “um, no, that didn’t happen.” And in particular, the “redundancy” of the oral tradition would prevent errors and falsehoods from spreading.
I just wanted to know what the data said about the reliability of oral tradition, and how easy/hard it is to manipulate memories on a mass scale.
You don’t have to convince millions of people all at once, in one generation, that they saw something, or even that their parents did.
You could, instead, convince a little cult today, and get that cult to grow over several generations until it is millions strong.
Or, instead, you could convince a small priestly caste that thus-and-so is a good legend to write down in what becomes the holy book; and teach millions of followers that everything written down in the priests’ holy book (which they can’t read) is true. Then some generations later, when there is an established tradition that whatever the priests’ book says, is true, when the masses finally do get to read the priests’ book, they will believe it too.
This is all obvious to me. It was pointed out in detail (along with other possibilities) during the debate, by myself and others. It was considered “unpersuasive.”
I actually made headway with one person (honestly he’s one of the most reasonable, fun to debate with guys I know) by saying “look, even if you consider the (fake story gets propogated somehow) idea impossible, divine miracles are ALSO impossible. So until we find more evidence, Mass Revelation can’t possibly be MORE likely than 50%.