It can be murky to infer what people believe based on actions or commitments, because this mixes two quantities: Probabilities and values. For example, the reason most elites don’t seem to take seriously efforts like shaping trajectories for strong AI is not because they think the probabilities of making a difference are astronomically small but because they don’t bite Pascalian bullets. Their utility functions are not linear. If your utility function is linear, this is a reason that your actions (if not your beliefs) will diverge from those of most elites. In any event, many elites are not even systematic or consequentialist in translating utilities times probabilities into actions.
I don’t endorse biting Pascalian bullets, in part for reasons argued in this post, which I think give further support to some considerations identified by GiveWell. In Pascalian cases, we have claims that people in general aren’t good at thinking about and which people generally assign low weight when they are acquainted with the arguments. I believe that Pascalian estimates of expected value that differ greatly from elite common sense and aren’t persuasive to elite common sense should be treated with great caution.
I also endorse Jonah’s point about some people caring about what you care about, but for different reasons. Just as we are weird, there can be other people who are weird in different ways that make them obsessed with the things we’re obsessed with for totally different reasons. Just as some scientists are obsessed with random stuff like dung beetles, I think a lot of asteroids were tracked because there were some scientists who are really obsessed with asteroids in particular, and want to ensure that all asteroids are carefully tracked far beyond the regular value that normal people place on tracking all the asteroids. I think this can include some borderline Pascalian issues. For example, important government agencies that care about speculative threats to national security. Dick Cheney famously said, “If there’s a 1% chance that Pakistani scientists are helping al-Qaeda build or develop a nuclear weapon, we have to treat it as a certainty in terms of our response.” Similarly, there can be people that are obsessed with many issues far out of proportion with what most ordinary people care about. Looking at what “most people” care about is less robust a way to find gaps in a market than it can appear at first. (I know you don’t think it would be good to save the world, but I think the example still illustrates the point to some extent. An example more relevant to would be that some scientists might just be really interested in insects and do a lot of the research that you’d think would be valuable, even though if you had just thought “no one cares about insects so this research will never happen” you’d be wrong.)
I don’t endorse biting Pascalian bullets, in part for reasons argued in this post, which I think give further support to some considerations identified by GiveWell.
As far as the GiveWell point, I meant “proper Pascalian bullets” where the probabilities are computed after constraining by some reasonable priors (keeping in mind that a normal distribution with mean 0 and variance 1 is not a reasonable prior in general).
In Pascalian cases, we have claims that people in general aren’t good at thinking about and which people generally assign low weight when they are acquainted with the arguments.
Low probability, yes, but not necessarily low probability*impact.
I believe that Pascalian estimates of expected value that differ greatly from elite common sense and aren’t persuasive to elite common sense should be treated with great caution.
As I mentioned in another comment, I think most Pascalian wagers that one comes across are fallacious because they miss even bigger Pascalian wagers that should be pursued instead. However, there are some Pascalian wagers that seem genuinely compelling even after looking for alternatives, like “the Overwhelming Importance of Shaping the Far Future.” My impression is that most elites do not agree that the far future is overwhelmingly important even after hearing your arguments because they don’t have linear utility functions and/or don’t like Pascalian wagers. Do you think most elites would agree with you about shaping the far future?
This highlights a meta-point in this discussion: Often what’s under debate here is not the framework but instead claims about (1) whether elites would or would not agree with a given position upon hearing it defended and (2) whether their sustained disagreement even after hearing it defended results from divergent facts, values, or methodologies (e.g., not being consequentialist). It can take time to assess these, so in the short term, disagreements about what elites would come to believe are a main bottleneck for using elite common sense to reach conclusions.
However, there are some Pascalian wagers that seem genuinely compelling even after looking for alternatives, like “the Overwhelming Importance of Shaping the Far Future.” My impression is that most elites do not agree that the far future is overwhelmingly important even after hearing your arguments because they don’t have linear utility functions and/or don’t like Pascalian wagers. Do you think most elites would agree with you about shaping the far future?
I disagree with the claim that the argument for shaping the far future is a Pascalian wager. In my opinion, there is a reasonably high, reasonably non-idiosyncratic probability that humanity will survive for a very long time, that there will be a lot of future people, and/or that future people will have a very high quality of life. Though I have not yet defended this claim as well as I would like, I also believe that many conventionally good things people can do push toward future generations facing future challenges and opportunities better than they otherwise would, which with a high enough and conventional enough probability makes the future go better. I think that these are claims which elite common sense would be convinced of, if in possession of my evidence. If elite common sense would not be so convinced, I would consider abandoning these assumptions.
Regarding the more purely moral claims, I suspect there are a wide variety of considerations which elite common sense would give weight to, and that very long-term considerations are one time of important consideration which would get weight according to elite common sense. It may also be, in part, a fundamental difference of values, where I am part of a not-too-small contingent of people who have distinctive concerns. However, in genuinely altruistic contexts, I think many people would give these considerations substantially more weight if they thought about the issue carefully.
Near the beginning of my dissertation, I actually speak about the level of confidence I have in my thesis quite tentatively:
How convinced should you be by the arguments I’m going to give? I’m defending an unconventional thesis and my support for that thesis comes from highly speculative arguments. I don’t have great confidence in my thesis, or claim that others should. But I am convinced that it could well be true, that the vast majority of thoughtful people give the claim less credence that they should, and that it is worth thinking about more carefully. I aim to make the reader justified in taking a similar attitude. (p. 3, Beckstead 2013)
I disagree with the claim that the argument for shaping the far future is a Pascalian wager.
I thought some of our disagreement might stem from understanding what each other meant, and that seems to have been true here. Even if the probability of humanity surviving a long time is large, there remain entropy in our influence and butterfly effects, such that it seems extremely unlikely that what we do now will actually make a pivotal difference in the long term, and we could easily be getting the sign wrong. This makes the probabilities small enough to seem Pascalian for most people.
It’s very common for people to say, “Predictions are hard, especially about the future, so let’s focus on the short term where we can be more confident we’re at least making a small positive impact.”
It’s very common for people to say, “Predictions are hard, especially about the future, so let’s focus on the short term where we can be more confident we’re at least making a small positive impact.”
If by short-term you mean “what happens in the next 100 years or so,” I think there is something to this idea, even for people who care primarily about very long-term considerations. I suspect it is true that the expected value of very long-run outcomes is primarily dominated by totally unforeseeable weird stuff that could happen in the distant future. But I believe that the best way deal with this challenge is to empower humanity to deal with the relatively foreseeable and unforeseeable challenges and opportunities that it will face over the next few generations. This doesn’t mean “let’s just look only at short-run well-being boosts,” but something more like “let’s broadly improve cooperation, motives, access to certain types of information, narrow and broad technological capabilities, and intelligence and rationality to deal with the problems we can’t foresee, and let’s rely on the best evidence we can to prepare for the problems we can foresee.” I say a few things about this issue here. I hope to say more about it in the future.
An analogy would be that if you were a 5-year-old kid and you primarily cared about how successful you were later in life, you should focus on self-improvement activities (like developing good habits, gaining knowledge, and learning how to interact with other people) and health and safety issues (like getting adequate nutrition, not getting hit by cars, not poisoning yourself, not falling off of tall objects, and not eating lead-based paint). You should not try to anticipate fine-grained challenges in the labor market when you graduate from college or disputes you might have with your spouse. I realize that this analogy may not be compelling, but perhaps it illuminates my perspective.
I don’t endorse biting Pascalian bullets, in part for reasons argued in this post, which I think give further support to some considerations identified by GiveWell. In Pascalian cases, we have claims that people in general aren’t good at thinking about and which people generally assign low weight when they are acquainted with the arguments. I believe that Pascalian estimates of expected value that differ greatly from elite common sense and aren’t persuasive to elite common sense should be treated with great caution.
I also endorse Jonah’s point about some people caring about what you care about, but for different reasons. Just as we are weird, there can be other people who are weird in different ways that make them obsessed with the things we’re obsessed with for totally different reasons. Just as some scientists are obsessed with random stuff like dung beetles, I think a lot of asteroids were tracked because there were some scientists who are really obsessed with asteroids in particular, and want to ensure that all asteroids are carefully tracked far beyond the regular value that normal people place on tracking all the asteroids. I think this can include some borderline Pascalian issues. For example, important government agencies that care about speculative threats to national security. Dick Cheney famously said, “If there’s a 1% chance that Pakistani scientists are helping al-Qaeda build or develop a nuclear weapon, we have to treat it as a certainty in terms of our response.” Similarly, there can be people that are obsessed with many issues far out of proportion with what most ordinary people care about. Looking at what “most people” care about is less robust a way to find gaps in a market than it can appear at first. (I know you don’t think it would be good to save the world, but I think the example still illustrates the point to some extent. An example more relevant to would be that some scientists might just be really interested in insects and do a lot of the research that you’d think would be valuable, even though if you had just thought “no one cares about insects so this research will never happen” you’d be wrong.)
As far as the GiveWell point, I meant “proper Pascalian bullets” where the probabilities are computed after constraining by some reasonable priors (keeping in mind that a normal distribution with mean 0 and variance 1 is not a reasonable prior in general).
Low probability, yes, but not necessarily low probability*impact.
As I mentioned in another comment, I think most Pascalian wagers that one comes across are fallacious because they miss even bigger Pascalian wagers that should be pursued instead. However, there are some Pascalian wagers that seem genuinely compelling even after looking for alternatives, like “the Overwhelming Importance of Shaping the Far Future.” My impression is that most elites do not agree that the far future is overwhelmingly important even after hearing your arguments because they don’t have linear utility functions and/or don’t like Pascalian wagers. Do you think most elites would agree with you about shaping the far future?
This highlights a meta-point in this discussion: Often what’s under debate here is not the framework but instead claims about (1) whether elites would or would not agree with a given position upon hearing it defended and (2) whether their sustained disagreement even after hearing it defended results from divergent facts, values, or methodologies (e.g., not being consequentialist). It can take time to assess these, so in the short term, disagreements about what elites would come to believe are a main bottleneck for using elite common sense to reach conclusions.
I disagree with the claim that the argument for shaping the far future is a Pascalian wager. In my opinion, there is a reasonably high, reasonably non-idiosyncratic probability that humanity will survive for a very long time, that there will be a lot of future people, and/or that future people will have a very high quality of life. Though I have not yet defended this claim as well as I would like, I also believe that many conventionally good things people can do push toward future generations facing future challenges and opportunities better than they otherwise would, which with a high enough and conventional enough probability makes the future go better. I think that these are claims which elite common sense would be convinced of, if in possession of my evidence. If elite common sense would not be so convinced, I would consider abandoning these assumptions.
Regarding the more purely moral claims, I suspect there are a wide variety of considerations which elite common sense would give weight to, and that very long-term considerations are one time of important consideration which would get weight according to elite common sense. It may also be, in part, a fundamental difference of values, where I am part of a not-too-small contingent of people who have distinctive concerns. However, in genuinely altruistic contexts, I think many people would give these considerations substantially more weight if they thought about the issue carefully.
Near the beginning of my dissertation, I actually speak about the level of confidence I have in my thesis quite tentatively:
I stand by this tentative stance.
I thought some of our disagreement might stem from understanding what each other meant, and that seems to have been true here. Even if the probability of humanity surviving a long time is large, there remain entropy in our influence and butterfly effects, such that it seems extremely unlikely that what we do now will actually make a pivotal difference in the long term, and we could easily be getting the sign wrong. This makes the probabilities small enough to seem Pascalian for most people.
It’s very common for people to say, “Predictions are hard, especially about the future, so let’s focus on the short term where we can be more confident we’re at least making a small positive impact.”
If by short-term you mean “what happens in the next 100 years or so,” I think there is something to this idea, even for people who care primarily about very long-term considerations. I suspect it is true that the expected value of very long-run outcomes is primarily dominated by totally unforeseeable weird stuff that could happen in the distant future. But I believe that the best way deal with this challenge is to empower humanity to deal with the relatively foreseeable and unforeseeable challenges and opportunities that it will face over the next few generations. This doesn’t mean “let’s just look only at short-run well-being boosts,” but something more like “let’s broadly improve cooperation, motives, access to certain types of information, narrow and broad technological capabilities, and intelligence and rationality to deal with the problems we can’t foresee, and let’s rely on the best evidence we can to prepare for the problems we can foresee.” I say a few things about this issue here. I hope to say more about it in the future.
An analogy would be that if you were a 5-year-old kid and you primarily cared about how successful you were later in life, you should focus on self-improvement activities (like developing good habits, gaining knowledge, and learning how to interact with other people) and health and safety issues (like getting adequate nutrition, not getting hit by cars, not poisoning yourself, not falling off of tall objects, and not eating lead-based paint). You should not try to anticipate fine-grained challenges in the labor market when you graduate from college or disputes you might have with your spouse. I realize that this analogy may not be compelling, but perhaps it illuminates my perspective.