’Tis remarkable how many disputes between would-be rationalists end in a game of reference class tennis. I suspect this is because our beliefs are partially driven by “intuition” (i.e. subcognitive black boxes giving us advice) (not that there’s anything wrong with that), and when it comes time to try and share our intuition with other minds, we try to point to cases that “look similar”, or the examples whereby our brain learned to pattern-recognize and judge “that sort” of case.
My own cached rule for such cases is to try and look inside the thing itself, rather than comparing it to other things—to drop into causal analysis, rather than trying to hit the ball back into your own preferred concept boundary of similar things. Focus on the object level, rather than the meta; and try to argue less by similarity, for the universe itself is not driven by Similarity and Contagion, after all.
Sometimes “looking at the thing itself” is too costly or too difficult. How can the proverbial “bright sixteen-year-old” sitting in a high school classroom figure out the truth about, say, the number of protons in an atom of gold, without having to accept the authority of his textbooks and instructors? If there were a bunch of well-funded nutcases dedicated to arguing that gold atoms have seventy-eight protons instead of seventy-nine, the only way you can really judge who’s correct is to judge the relative credibility of the people presenting the evidence. After all, one side’s evidence could be completely fraudulent and you’d have no way of knowing that.
Far too often, reference classes and meta-level discussions are all we have.
Then let us try to figure out whose authority is to be trusted about experimental results and work from there. Cases where you can reduce it to a direct conflict about easily observable facts, and then work from there, are much more likely to have one dramatically more trustworthy party.
How should we unpack black boxes we don’t have yet? For example a non-neural language capable self-maintaining goal-oriented system*.
We have a surfeit of potential systems (with different capabilities of self-inspection and self-modification) with no way to test whether they will fall into the above category or how big the category actually is.
I estimate that even fairly bad reference class / outside view analysis is still far more reliable than the best inside view that can be realistically expected. People are just spectacularly bad at inside view analysis, and reference class analysis puts hard boundaries within which truth is almost always found.
If I may attempt to summarize the link: Eliezer maintains that, while the quantitative inside view is likely to fail in cases where the underlying causes are not understood or planning biases are likely to be in effect, the outside view cannot be expected to work when the underlying causes undergo sufficiently severe alterations. Rather, he proposes what he calls the “weak inside view”—an analysis of underlying causes noting the most extreme of changes and stating qualitatively their consequences.
Is there any evidence that in cases that where neither “outside view” nor “strong inside view” can be applied, “weak inside view” is at least considerably better than pure chance? I have strong doubts about it.
Yes, it would be good to have a clearer data set of topics at dates, the views suggested by different styles of analysis, and what we think now about who was right. I’m pretty skeptical about this weak inside view claim, but will defer to some more systematic data. Of course that is my suggesting we take an outside view to evaluating this claim about which view is more reliable.
’Tis remarkable how many disputes between would-be rationalists end in a game of reference class tennis. I suspect this is because our beliefs are partially driven by “intuition” (i.e. subcognitive black boxes giving us advice) (not that there’s anything wrong with that), and when it comes time to try and share our intuition with other minds, we try to point to cases that “look similar”, or the examples whereby our brain learned to pattern-recognize and judge “that sort” of case.
My own cached rule for such cases is to try and look inside the thing itself, rather than comparing it to other things—to drop into causal analysis, rather than trying to hit the ball back into your own preferred concept boundary of similar things. Focus on the object level, rather than the meta; and try to argue less by similarity, for the universe itself is not driven by Similarity and Contagion, after all.
Sometimes “looking at the thing itself” is too costly or too difficult. How can the proverbial “bright sixteen-year-old” sitting in a high school classroom figure out the truth about, say, the number of protons in an atom of gold, without having to accept the authority of his textbooks and instructors? If there were a bunch of well-funded nutcases dedicated to arguing that gold atoms have seventy-eight protons instead of seventy-nine, the only way you can really judge who’s correct is to judge the relative credibility of the people presenting the evidence. After all, one side’s evidence could be completely fraudulent and you’d have no way of knowing that.
Far too often, reference classes and meta-level discussions are all we have.
Then let us try to figure out whose authority is to be trusted about experimental results and work from there. Cases where you can reduce it to a direct conflict about easily observable facts, and then work from there, are much more likely to have one dramatically more trustworthy party.
How should we unpack black boxes we don’t have yet? For example a non-neural language capable self-maintaining goal-oriented system*.
We have a surfeit of potential systems (with different capabilities of self-inspection and self-modification) with no way to test whether they will fall into the above category or how big the category actually is.
*I’m trying to unpack AGI here somewhat
I estimate that even fairly bad reference class / outside view analysis is still far more reliable than the best inside view that can be realistically expected. People are just spectacularly bad at inside view analysis, and reference class analysis puts hard boundaries within which truth is almost always found.
http://lesswrong.com/lw/vz/the_weak_inside_view/
If I may attempt to summarize the link: Eliezer maintains that, while the quantitative inside view is likely to fail in cases where the underlying causes are not understood or planning biases are likely to be in effect, the outside view cannot be expected to work when the underlying causes undergo sufficiently severe alterations. Rather, he proposes what he calls the “weak inside view”—an analysis of underlying causes noting the most extreme of changes and stating qualitatively their consequences.
Is there any evidence that in cases that where neither “outside view” nor “strong inside view” can be applied, “weak inside view” is at least considerably better than pure chance? I have strong doubts about it.
Yes, it would be good to have a clearer data set of topics at dates, the views suggested by different styles of analysis, and what we think now about who was right. I’m pretty skeptical about this weak inside view claim, but will defer to some more systematic data. Of course that is my suggesting we take an outside view to evaluating this claim about which view is more reliable.