I’m not certain that the Sino Forest ‘case’ is a very good one for use here, from reading the lw’er’s “tl;dr” version of the full article.
Consider: While the “source” of information should not be used to evaluate the probability of that information’s accuracy, the manner in which information is delivered should. If an improper format of accreditation is used; if obvious and overt logical fallacies are present—that information would rightly be discounted from any estimate of the accuracy of a statement.
For example; I consider myself a strong/gnostic atheist. So for me, it is obvious that “atheism” is the “correct” answer. Should I encounter someone, however, whose sole reason for being an atheist was because their little sister died (WARNING: TVTROPES.org link!!) -- I would be forced to tell that person ve was being a complete moron, and that nobody should listen to ver reasoning.
The mere accident of a conclusion coinciding with the truth does not make that conclusion justified. As part of the assessment tools of judging the accreditation weighting we provide any given input, the ‘proper indicators of subject-expertise’ should weigh in highly. If a hobo with a speech impediment walks up to me on the street and tells me he’s got a surefire plan for making money on the stock-market written on the back of a Subway’s wrapper, I’m not going to be inclined to waste my time reading it. On the other hand, if a man makes the news from raking in billions over the course of a week from mere thousands and then offers me a secret algorithm that’s the source of his wealth, I’m going to take him quite seriously: because he’s got the right indicators of veracity of his beliefs.
Now, that being said—there needs to be a means to re-weight assessments of credulity of statemets to account for outlier positions. I would tend to believe that willingness to publicize resources and citing credible sources of data, rigorousness fo documentation of ‘radical’ claims, and so one would be useful to this end.
As Sagan said: “Extraordinary claims require extraordinary evidence.” While we should remember that’s just a heuristic—I don’t know that I see a problem here. Limited information and constant need for reassessment based on it means mistakes will be made. They are unavoidable. Do not let the perfect become the enemy of the good.
Reading the links, the one mentioned error by the whistleblower was actually just an example of real data contradicting the official lies. All the other mentioned criticisms are one of surface style.
What I personally found most interesting about this was not that the whistleblowers were discounted before and after they were proven right (we see this in many bubbles, for example, the housing bubble, and apparently it’s not yet certain that Sino Forestry is a massive fraud), but how one could use a sort of Outside View/Fermi calculation to sanity-check the claims. If Sino Forestry was really causing 17m cubic meters of wood to be processed a year, where was all the processing? This basic simple question tells us a lot.
I’m personally wary of hindsight bias sort of stuff. Of course this all seems obvious to us now, but… would it have when we first examined it? I don’t know.
As far as Sino Forestry goes; I haven’t read the full analysis published that triggered it, but nothing in what I’ve read has mentioned hot new data which makes the fraud case convincing.
I’m not certain that the Sino Forest ‘case’ is a very good one for use here, from reading the lw’er’s “tl;dr” version of the full article.
Consider: While the “source” of information should not be used to evaluate the probability of that information’s accuracy, the manner in which information is delivered should. If an improper format of accreditation is used; if obvious and overt logical fallacies are present—that information would rightly be discounted from any estimate of the accuracy of a statement.
For example; I consider myself a strong/gnostic atheist. So for me, it is obvious that “atheism” is the “correct” answer. Should I encounter someone, however, whose sole reason for being an atheist was because their little sister died (WARNING: TVTROPES.org link!!) -- I would be forced to tell that person ve was being a complete moron, and that nobody should listen to ver reasoning.
The mere accident of a conclusion coinciding with the truth does not make that conclusion justified. As part of the assessment tools of judging the accreditation weighting we provide any given input, the ‘proper indicators of subject-expertise’ should weigh in highly. If a hobo with a speech impediment walks up to me on the street and tells me he’s got a surefire plan for making money on the stock-market written on the back of a Subway’s wrapper, I’m not going to be inclined to waste my time reading it. On the other hand, if a man makes the news from raking in billions over the course of a week from mere thousands and then offers me a secret algorithm that’s the source of his wealth, I’m going to take him quite seriously: because he’s got the right indicators of veracity of his beliefs.
Now, that being said—there needs to be a means to re-weight assessments of credulity of statemets to account for outlier positions. I would tend to believe that willingness to publicize resources and citing credible sources of data, rigorousness fo documentation of ‘radical’ claims, and so one would be useful to this end.
As Sagan said: “Extraordinary claims require extraordinary evidence.” While we should remember that’s just a heuristic—I don’t know that I see a problem here. Limited information and constant need for reassessment based on it means mistakes will be made. They are unavoidable. Do not let the perfect become the enemy of the good.
Reading the links, the one mentioned error by the whistleblower was actually just an example of real data contradicting the official lies. All the other mentioned criticisms are one of surface style.
What I personally found most interesting about this was not that the whistleblowers were discounted before and after they were proven right (we see this in many bubbles, for example, the housing bubble, and apparently it’s not yet certain that Sino Forestry is a massive fraud), but how one could use a sort of Outside View/Fermi calculation to sanity-check the claims. If Sino Forestry was really causing 17m cubic meters of wood to be processed a year, where was all the processing? This basic simple question tells us a lot.
I’m personally wary of hindsight bias sort of stuff. Of course this all seems obvious to us now, but… would it have when we first examined it? I don’t know.
That question reminds me of http://www.gladwell.com/2007/2007_01_08_a_secrets.html
As far as Sino Forestry goes; I haven’t read the full analysis published that triggered it, but nothing in what I’ve read has mentioned hot new data which makes the fraud case convincing.