Hmm… That might temper my view a bit. Still, though, I think a non-standard set of experts qualifies as an inside view. For example, Inadequate Equilibria mentions trusting Scott Sumner instead of the Central Bank of Japan, and another example might be something like “all professors and investors I’ve spoken to think my startup idea is stupid, but my roommate and co-founder still really believes in it”.
I’d object to the phrase ‘non-standard’. Really, if maths journals count as ‘standard set of experts’ in Modest Epistemology (TM Greg) then Modest Epistemology (TM Greg) is sneaking in a lot of extra empirical claims—it takes work to decide to trust the academic field of math to be the best in the world at figuring out true math, and is not a primitive of one’s worldview.
You have to look at where math advances have come from historically, you have to look at incentives, you have to look at other players, and then come to the (broadly correct belief) that academic math journals are the best place to get math truth from—and then my arguments for MIRI having an edge in this market can interface with your reasons. There’s no free lunch of experts to trust, it’s personal models all the way down.
One of the fundamental things wrong with Modest Epistemology (TM Greg) is that Greg’s post appears to imply that figuring out who are experts and what their opinions are is in some way trivial. I think the alternative is not “if you think experts are wrong that’s fine” but “please dear god can we build some models of why and when to trust different groups of people to figure out different things”. The correct response is not to reject experts, but to actually practice trying to figure out when to trust them, so that you can notice inefficiencies and biases in the market and make a profit / find free value.
It felt to me like a post about a largely empirical discussion was instead 90% abstract theory—very interesting and useful theory, but it wasn’t the topic of disagreement (to my eyes).
I agree with your conclusion, that the important takeaway is to build models of whom to trust when and on what matters.
Nonetheless, I disagree that it requires as much work to decide to trust the academic field of math as to trust MIRI. Whenever you’re using the outside view, you need to define a reference class. I’ve never come across this used as an objection to the outside view. That’s probably because there often is one such class more salient than others: “people who have social status within field X”. After all, one of the key explanations for the evolution of human intelligence is that of an arms race in social cognition. For example, you see this in studies where people are clueless at solving logic problems, unless you phrase them in terms of detecting cheaters or other breaches of social contracts (see e.g. Cheng & Holyoak, 1985 and Gigerenzer & Hug, 1992). So we should expect humans to easily figure out who has status within a field, but to have a very hard time figuring out who gets the field closer to truth.
Isn’t this exactly why modesty is such an appealing and powerful view in the first place? Because choosing the reference class is so easy (not requiring much object-level investigation), and experts are correct sufficiently often, that any inside view is mistaken in expectation.
Right. I will say that the heuristic of trusting “people who have social status within field X” seems to have very obvious flaws and biases, and it’s important to build models to work past those mistakes. Furthermore, in the world around us, most high status people in institutions are messing up most of the time. Bank of Japan is fine example, but also the people running my prestigious high school, my university, researchers in my field, etc. It’s important to learn where they’re inadequate so you can find extra value.
I’m not saying that in general “trust the high status people” isn’t probably okay if you have zero further info, but you definitely need to start building detailed models and get better than that otherwise you’re definitely not going to save the world.
Hmm… That might temper my view a bit. Still, though, I think a non-standard set of experts qualifies as an inside view. For example, Inadequate Equilibria mentions trusting Scott Sumner instead of the Central Bank of Japan, and another example might be something like “all professors and investors I’ve spoken to think my startup idea is stupid, but my roommate and co-founder still really believes in it”.
I’d object to the phrase ‘non-standard’. Really, if maths journals count as ‘standard set of experts’ in Modest Epistemology (TM Greg) then Modest Epistemology (TM Greg) is sneaking in a lot of extra empirical claims—it takes work to decide to trust the academic field of math to be the best in the world at figuring out true math, and is not a primitive of one’s worldview.
You have to look at where math advances have come from historically, you have to look at incentives, you have to look at other players, and then come to the (broadly correct belief) that academic math journals are the best place to get math truth from—and then my arguments for MIRI having an edge in this market can interface with your reasons. There’s no free lunch of experts to trust, it’s personal models all the way down.
One of the fundamental things wrong with Modest Epistemology (TM Greg) is that Greg’s post appears to imply that figuring out who are experts and what their opinions are is in some way trivial. I think the alternative is not “if you think experts are wrong that’s fine” but “please dear god can we build some models of why and when to trust different groups of people to figure out different things”. The correct response is not to reject experts, but to actually practice trying to figure out when to trust them, so that you can notice inefficiencies and biases in the market and make a profit / find free value.
It felt to me like a post about a largely empirical discussion was instead 90% abstract theory—very interesting and useful theory, but it wasn’t the topic of disagreement (to my eyes).
I agree with your conclusion, that the important takeaway is to build models of whom to trust when and on what matters.
Nonetheless, I disagree that it requires as much work to decide to trust the academic field of math as to trust MIRI. Whenever you’re using the outside view, you need to define a reference class. I’ve never come across this used as an objection to the outside view. That’s probably because there often is one such class more salient than others: “people who have social status within field X”. After all, one of the key explanations for the evolution of human intelligence is that of an arms race in social cognition. For example, you see this in studies where people are clueless at solving logic problems, unless you phrase them in terms of detecting cheaters or other breaches of social contracts (see e.g. Cheng & Holyoak, 1985 and Gigerenzer & Hug, 1992). So we should expect humans to easily figure out who has status within a field, but to have a very hard time figuring out who gets the field closer to truth.
Isn’t this exactly why modesty is such an appealing and powerful view in the first place? Because choosing the reference class is so easy (not requiring much object-level investigation), and experts are correct sufficiently often, that any inside view is mistaken in expectation.
Right. I will say that the heuristic of trusting “people who have social status within field X” seems to have very obvious flaws and biases, and it’s important to build models to work past those mistakes. Furthermore, in the world around us, most high status people in institutions are messing up most of the time. Bank of Japan is fine example, but also the people running my prestigious high school, my university, researchers in my field, etc. It’s important to learn where they’re inadequate so you can find extra value.
I’m not saying that in general “trust the high status people” isn’t probably okay if you have zero further info, but you definitely need to start building detailed models and get better than that otherwise you’re definitely not going to save the world.