Brian_Tomasik

Karma: 365

Brian_Tomasik 8 Jun 2013 20:38 UTC
4 points
on: Social Impact, Effective Altruism, and Motivated Cognition

For now, I would encourage effective altruist types to take pride in being self-skeptical when it comes to favorable assessments of their potential impact relative to other effective altruist types, or relative to people outside of the effective altruist community.

Yes, I find it remarkable how EAs tend to think their work is obviously vastly more important than that of “non-EAs” (as if such a thing were even well defined). There’s not a lot new under the sun, and like most movements, EA is largely a recycling and recombination of things other people have been doing since the dawn of civilization. It may be a good combination, but little in EA is really unique to EA.

All of that said, I think a big reason people think their own work dominates that of others is because they have different values from other people. It’s perfectly possible for lots of people to be doing lots of things that are each optimal relative to their own values. You might (perhaps correctly) point out that most EAs have values more similar to each other than my values are to theirs, so my point may apply less broadly than I suggested.

Brian_Tomasik 8 Jun 2013 21:03 UTC
2 points
in reply to: JonahS’s comment on: Many Weak Arguments vs. One Relatively Strong Argument
Cool. Yes, many examples of #1 come to mind. As far as #2, I don’t believe I had thought of this as a principle specifically.

What I meant about a single factor dominating in physics was that in most cases, even when multiple factors are at play, one of those factors matters more than all the rest such that you can ignore the rest. For example, an electron has gravitational attraction to the atomic nucleus, but this is trivial compared with the electromagnetic attraction. Similarly, the electromagnetic repulsion of the protons in the nucleus is trivial compared with the strong force holding them together. It’s rare in nature to have a close competition between competing forces, at least until you get to higher-level domains like inter-agent competition.

Brian_Tomasik 13 Jun 2013 5:17 UTC
4 points
in reply to: Peter Wildeford’s comment on: Effective Altruism Through Advertising Vegetarianism?
Good points, Carl! Jonah Sinick actually made the GOTV argument to me on a prior occasion, citing your essay on the topic.

One additional consideration is that nearly everyone knows about voting, but many people don’t know about the cruelty of factory farms. This goes along with the low-hanging-fruit point.

I would not be surprised if, after tempering the figures by this outside-view prior, it takes a few hundred dollars to create a new veg year. Even if so, that’s at most 1-2 orders of magnitude different from the naive conservative estimate.

Brian_Tomasik 3 Aug 2013 10:05 UTC
1 point
in reply to: CarlShulman’s comment on: Four Focus Areas of Effective Altruism
The 2-50 bounds seem reasonable for EV(happiness)/EV(suffering) using normal people’s pleasure-pain exchange ratios, which I think are insane. :) Something like accepting 1 minute of Medieval torture for a few days/weeks of good life.

Using my own pleasure-pain exchange ratio, the future is almost guaranteed to be negative in expectation, although I maintain ~30% chance that it would still be better for Earth-based life to colonize space to prevent counterfactual suffering elsewhere.

Brian_Tomasik 11 Aug 2013 23:02 UTC
4 points
on: Common sense as a prior
Great post, Nick! I agree with most of what you say, although there are times when I don’t always demonstrate this in practice. Your post is what I would consider a good “motivational speech”—an eloquent defense of something you agree with but could use reminding of on occasion.

It’s good to get outside one’s intellectual bubble, even a bubble as fascinating and sophisticated as LessWrong. Even on the seemingly most obvious of questions, we could be making logical mistakes.

I think the focus on only intellectual elites has unclear grounding. Is the reason because elites think most seriously about the questions that you care about most? On a question of which kind of truck was most suitable for garbage collection, you would defer to a different class of people. In such a case, I guess you would regard them as the (question-dependent) “elites.”

It can be murky to infer what people believe based on actions or commitments, because this mixes two quantities: Probabilities and values. For example, the reason most elites don’t seem to take seriously efforts like shaping trajectories for strong AI is not because they think the probabilities of making a difference are astronomically small but because they don’t bite Pascalian bullets. Their utility functions are not linear. If your utility function is linear, this is a reason that your actions (if not your beliefs) will diverge from those of most elites. In any event, many elites are not even systematic or consequentialist in translating utilities times probabilities into actions.

Insofar as my own actions are atypical, I intend for it to result from atypical moral beliefs rather than atypical factual beliefs. (If you can think of instances of clearly atypical factual beliefs on my part, let me know.) Of course, you could claim, elite common sense should apply also as a prior to what my own moral beliefs actually are, given the fallibility of introspection. This is true, but its importance depends on how abstractly I view my own moral values. If I ask questions about what an extrapolated Brian would think upon learning more, having more experiences, etc., then the elite prior has a lot to say on this question. But if I’m more concerned with my very immediate emotional reaction, then there’s less room for error and less that the common-sense prior has to say. The fact that my moral values are sometimes not strongly affected by common-sense moral values comes from my favoring immediate emotions rather than what (one of many possible) extrapolated Brians would feel upon having further and different life experiences. (Of course, there are many possible further life experiences I could have, which would push me in lots of random directions. This is why I’m not so gung ho about what my extrapolated selves would think on some questions.)

Finally, as you point out, it can be useful to make contrarian points for the purpose of intellectual progress. Most startups, experiments, and new theories fail, and you’re more likely to be right by sticking with conventional wisdom than betting on something new. Yet if no one tried new things and pushed the envelope, we’d have an epistemic “tragedy of the commons” where everyone tries to make her own views more accurate at the cost of slowing the overall intellectual progress of society. That said, we can sometimes explore weird ideas without actually betting on them when the stakes are high, although sometimes (as in the case of startups), you do have to take on high risks. Maybe there would be fewer startups if the founders were more sober-minded in their assessment of their odds of success.

Brian_Tomasik 12 Aug 2013 5:14 UTC
3 points
in reply to: JonahS’s comment on: Common sense as a prior
Fair enough. :) Yes, from the fact that probability * utility is small, we can’t tell whether the probability is small or the utility is, or both. In the case of shaping AI specifically, I haven’t heard good arguments against assigning it non-negligible probability of success, and I also know that many people don’t bite Pascalian wagers at least partly because they don’t like Pascalian wagers rather than because they disagree with the premises, so combining these suggests the probability side isn’t so much the issue, but this suggestion stands to be verified. Also, people will often feign having ridiculously small probabilities to get out of Pascalian wagers, but they usually make these proclamations after the fact, or else are the kind of people who say “any probability less than 0.01 is set to 0” (except when wearing seat belts to protect against a car accident or something, highlighting what Nick said about people potentially being more rational for important near-range decisions).

Anyway, not accepting a Pascalian wager does not mean you don’t agree with the probability and utility estimates; maybe you think the wager is missing the forest for the trees and ignoring bigger-picture issues. I think most Pascalian wagers can be defused by saying, “If that were true, this other thing would be even more important, so you should focus on that other thing instead.” But then you should actually focus on that other thing instead rather than focusing on neither, which most people tend to do. :P

You are also correct that differences in moral values doesn’t completely shield off an update to probabilities when I find my actions divergent from those of others. However, in cases when people do make their probabilities explicit, I don’t normally diverge substantially (or if I do, I tend to update somewhat), and in these particular cases, divergent values comprise the remainder of the gap (usually most of it). Of course, I may have already updated the most in those cases where people have made their probabilities explicit, so maybe there’s bigger latent epistemic divergence when we’re distant from the lamp post.
What links here?
- Brian_Tomasik's comment on Common sense as a prior by Nick_Beckstead (12 Aug 2013 19:30 UTC; 2 points)

Brian_Tomasik 12 Aug 2013 17:06 UTC
0 points
in reply to: Jiro’s comment on: Common sense as a prior
Yep, that’s right. In my top-level comment, I said, “In any event, many elites are not even systematic or consequentialist in translating utilities times probabilities into actions.” Still, on big government-policy questions that affect society (rather than personal actions, relationships, etc.) elites tend to be (relatively) more interested in utilitarian calculations.

Brian_Tomasik 12 Aug 2013 17:39 UTC
6 points
in reply to: Nick_Beckstead’s comment on: Common sense as a prior
Cool—that makes sense. In principle, would you still count everyone with some (possibly very small) weight, the way PageRank does? (Or maybe negative weight in a few cases.) A binary separation between elites and non-elites seems hacky, though of course, it may in fact be best to do so in practice to make the analysis tractable. Cutting out part of the sample also leads to a biased estimator, but maybe that’s not such a big deal in most cases if the weight on the remaining part was small anyway. You could also give different weight among the elites. Basically, elites vs. non-elites is a binary approximation of a more continuous weighting distribution. Anyway, it may be misleading to think of this as purely a weighted sample of opinion, because (a) you want to reduce the weight of beliefs that are copied from each other and (b) you may want to harmonize the beliefs in a way that’s different from blind averaging. Also, as you suggested, (c) you may want to dampen outliers to avoid pulling the average too much toward the outlier.

Brian_Tomasik 12 Aug 2013 19:30 UTC
2 points
in reply to: Nick_Beckstead’s comment on: Common sense as a prior

I don’t endorse biting Pascalian bullets, in part for reasons argued in this post, which I think give further support to some considerations identified by GiveWell.

As far as the GiveWell point, I meant “proper Pascalian bullets” where the probabilities are computed after constraining by some reasonable priors (keeping in mind that a normal distribution with mean 0 and variance 1 is not a reasonable prior in general).

In Pascalian cases, we have claims that people in general aren’t good at thinking about and which people generally assign low weight when they are acquainted with the arguments.

Low probability, yes, but not necessarily low probability*impact.

I believe that Pascalian estimates of expected value that differ greatly from elite common sense and aren’t persuasive to elite common sense should be treated with great caution.

As I mentioned in another comment, I think most Pascalian wagers that one comes across are fallacious because they miss even bigger Pascalian wagers that should be pursued instead. However, there are some Pascalian wagers that seem genuinely compelling even after looking for alternatives, like “the Overwhelming Importance of Shaping the Far Future.” My impression is that most elites do not agree that the far future is overwhelmingly important even after hearing your arguments because they don’t have linear utility functions and/or don’t like Pascalian wagers. Do you think most elites would agree with you about shaping the far future?

This highlights a meta-point in this discussion: Often what’s under debate here is not the framework but instead claims about (1) whether elites would or would not agree with a given position upon hearing it defended and (2) whether their sustained disagreement even after hearing it defended results from divergent facts, values, or methodologies (e.g., not being consequentialist). It can take time to assess these, so in the short term, disagreements about what elites would come to believe are a main bottleneck for using elite common sense to reach conclusions.

Brian_Tomasik 12 Aug 2013 19:39 UTC
0 points
in reply to: Nick_Beckstead’s comment on: Common sense as a prior

I don’t think your specific version of anti-realism, or your philosophical perspective which says there is no real question here, are views which can command the assent of a broad coalition of trustworthy people.

My current meta-ethical view says I care about factual but not necessarily moral disagreements with respect to elites. One’s choice of meta-ethics is itself a moral decision, not a factual one, so this disagreement doesn’t much concern me. Of course, there are some places where I could be factually wrong in my meta-ethics, like with the logical reasoning in this comment, but I think most elites don’t think there’s something wrong with my logic, just something (ethically) wrong with my moral stance. Let me know if you disagree with this. Even with moral realists, I’ve never heard someone argue that it’s a factual mistake not to care about moral truth (what could that even mean?), just that it would be a moral mistake or an error of reasonableness or something like that.

Brian_Tomasik 12 Aug 2013 20:19 UTC
0 points
in reply to: JonahS’s comment on: Common sense as a prior
Thanks, Jonah. :)

If you restrict yourself to thoughtful, intelligent people who care about having a big positive impact on global welfare (which is a group substantially larger than the EA community)

But it’s a smaller group than the set of elites used for the common-sense prior. Hence, many elites don’t share our values even by this basic measure.

Of course, this is relative to more conventional values than utilitarianism, and so lots of their efforts go into things that aren’t utilitarian.

Yes, this was my point.

You may argue that the problems that they’re working on are different from the problems that you’re interested in addressing, but there may be strong analogies between the situations, and so their knowledge may be transferable.

Definitely. I wouldn’t claim otherwise.

I wouldn’t take people’s low focus on AI risk reduction as very strong evidence that people don’t care about astronomical waste.

In isolation, their not working on astronomical waste is not sufficient proof that their utility functions are not linear. However, combined with everything else I know about people’s psychology, it seems very plausible that they in fact don’t have linear utility functions.

Compare with behavioral economics. You can explain away any given discrepancy from classical microeconomic behavior by rational agents through an epicycle in the theory, but combined with all that we know about people’s psychology, we have reason to think that psychological biases themselves are playing a role in the deviations.

Carl has argued against this, but Peter Thiel is very sophisticated and so his view can’t be dismissed out of hand.

Not dismissed out of hand, but downweighted a fair amount. I think Carl is more likely to be right than Thiel on an arbitrary question where Carl has studied it and Thiel has not. Famous people are busy. Comments they make in an offhand way may be circulated in the media. Thiel has some good general intuition, sure, but his speculations on a given social trend don’t compare with more systematic research done by someone like Carl.

Brian_Tomasik 13 Aug 2013 0:56 UTC
1 point
in reply to: Nick_Beckstead’s comment on: Common sense as a prior

In my opinion, the history of philosophy is filled with a lot of people often smarter than you and me going around saying that their perspective is the unique one that solves everything and that other people are incoherent and so on.

I think it’s fair to say that concepts like libertarian free will and dualism in philosophy of mind are either incoherent or extremely implausible, though maybe the elite-common-sense prior would make us less certain of that than most on LessWrong seem to be.

Like Luke Muehlhauser, I believe that we don’t even know what we’re asking when we ask ethical questions

Yes, I think most of the confusion on this subject comes from disputing definitions. Luke says: “Within 20 seconds of arguing about the definition of ‘desire’, someone will say, ‘Screw it. Taboo ‘desire’ so we can argue about facts and anticipations, not definitions.’”

Here I would say, “Screw ethics and meta-ethics. All I’m saying is I want to do what I feel like doing, even if you and other elites don’t agree with it.”

I personally suspect your error lies in not considering the problem from perspectives other than “what does Brian Tomasik care about right now?”.

Sure, but this is not a factual error, just an error in being a reasonable person or something. :)

I should point out that “doing what I feel like doing” doesn’t necessarily mean running roughshod over other people’s values. I think it’s generally better to seek compromise and remain friendly to those with whom you want to cooperate. It’s just that this is an instrumental concession, not because I actually agree with the values that I’m willing to be nice to.

Brian_Tomasik 13 Aug 2013 1:04 UTC
2 points
in reply to: Nick_Beckstead’s comment on: Common sense as a prior

I disagree with the claim that the argument for shaping the far future is a Pascalian wager.

I thought some of our disagreement might stem from understanding what each other meant, and that seems to have been true here. Even if the probability of humanity surviving a long time is large, there remain entropy in our influence and butterfly effects, such that it seems extremely unlikely that what we do now will actually make a pivotal difference in the long term, and we could easily be getting the sign wrong. This makes the probabilities small enough to seem Pascalian for most people.

It’s very common for people to say, “Predictions are hard, especially about the future, so let’s focus on the short term where we can be more confident we’re at least making a small positive impact.”

Brian_Tomasik 13 Aug 2013 1:13 UTC
1 point
in reply to: JonahS’s comment on: Common sense as a prior

Elite common sense says that voting is important for altruistic reasons.

This is because they’re deontologists, not because they’re consequentialists with a linear utility function. So rather than suggesting more similarity in values, it suggests less. (That said, there’s more overlap between deontology and consequentialism than meets the eye.)

So I think it’s best to remain agnostic as to the degree to which variance in the humanitarian endeavors that people engage in is driven by variance in their values.

It may be best to examine on a case-by-case basis. We don’t need to just look at what people are doing and make inferences; we can also look at other psychological hints about how they feel regarding a given issue. Nick did suggest giving greater weight to what people believe (or, in this case, what they do) than their stated reasons for those beliefs (or actions), but he acknowledges this recommendation is controversial (e.g., Ray Dalio disagrees), and on some issues it seems like there’s enough other information to outweigh whatever inferences we might draw from actions alone. For example, we know people tend to be irrational in the religious domain based on other facts and so can somewhat discount the observed behavior there.

Points taken on the other issues we discussed.

Brian_Tomasik 13 Aug 2013 1:27 UTC
1 point
in reply to: Nick_Beckstead’s comment on: Common sense as a prior
Nick, what do you do about the Pope getting extremely high PageRank by your measure? You could say that most people who trust his judgment aren’t elites themselves, but some certainly are (e.g., heads of state, CEOs, celebrities). Every president in US history has given very high credence to the moral teachings of Jesus, and some have even given high credence to his factual teachings. Hitler had very high PageRank during the 1930s, though I guess he doesn’t now, and you could say that any algorithm makes mistakes some of the time.

ETA: I guess you did say in your post that we should be less reliant on elite common sense in areas like religion and politics where rationality is less prized. But I feel like a similar thing could be said to some extent of debates about moral conclusions. The cleanest area of application for elite common-sense is with respect to verifiable factual claims.

Brian_Tomasik 13 Aug 2013 4:18 UTC
3 points
in reply to: JonahS’s comment on: Common sense as a prior
Oh, definitely. The consequentialist justification only happens in obscure corners of geekdom like LessWrong and stat / poli sci journals.

Just ask people why they vote, and most of them will say things like “It’s a civic duty,” “Our forefathers died for this, so we shouldn’t waste it,” “If everyone didn’t vote, things would be bad,” …

I Googled the question and found similar responses in this article:

One reason that people often offer for voting is “But what if everybody thought that way?” [...]

Another reason for voting, offered by political scientists and lay individuals alike, is that it is a civic duty of every citizen in a democratic country to vote in elections. It’s not about trying to affect the electoral outcome; it’s about doing your duty as a democratic citizen by voting in elections.

Interestingly, the author also says: “Your decision to vote or not will not affect whether or not other people will vote (unless you are a highly influential person and you announce your voting intention to the world in advance of the election).” This may be mostly true in practice, but not in the limit as everyone approaches identity with you. It seems like this author is a two-boxer based on his statements. He calls timeless considerations “magical thinking.”

Brian_Tomasik 13 Aug 2013 5:33 UTC
3 points
in reply to: wedrifid’s comment on: Common sense as a prior
Thanks! I’m not sure I understood your comment. Did you mean that if the other agents aren’t similar enough to you, it’s an error to assume that your cooperating will cause them to cooperate?

I was drawing the inference about two-boxing from the fact that the author seemed to dismiss the possibility that what you do could possibly affect what others do in any circumstance.

Brian_Tomasik 13 Aug 2013 5:49 UTC
1 point
in reply to: wedrifid’s comment on: Common sense as a prior
Even among humans, there’s something to timeless considerations, right? If you were in a real prisoner’s dilemma with someone you didn’t know but who was very similar to you and had read a lot of the same things, it seems plausible you should cooperate? I don’t claim the effect is strong enough to operate in the realm of voting most of the time, but theoretically timeless considerations can matter for less-than-perfect copies of yourself.

Brian_Tomasik 13 Aug 2013 17:41 UTC
0 points
in reply to: JonahS’s comment on: Common sense as a prior
Do you have in mind average people, or, e.g., top 10% Ivy Leaguers … ?

Both. Remember that many Ivy Leaguers are liberal-arts majors. Even many that are quantitatively oriented I suspect aren’t familiar with the literature. I guess it takes a certain level of sophistication to think that voting doesn’t make a difference in expectation, so maybe most people fall into the bucket of those who haven’t really thought about the matter rigorously at all. (Remember, we’re including English and Art majors here.)

You could say, “If they knew the arguments, they would be persuaded,” which may be true, but that doesn’t explain why they already vote without knowing the arguments. Explaining that suggests deontology as a candidate hypothesis.

These reasons aren’t obviously deontological (even though they might sound like they are on first hearing).
- “It’s a civic duty” is deontological if anything is, because deontology is duty-based ethics.
- “If everyone didn’t vote, things would be bad” is an application of Kant’s categorical imperative.
- “Our forefathers died for this, so we shouldn’t waste it” is not deontological—just the sunk-cost fallacy.
Even if people did explicitly describe their reasons as deontological, one still wouldn’t know whether this was the case, because people’s stated reasons are often different from their actual reasons.

At some point it may become a debate about the teleological level at which you assess their “reasons.” As individuals, it’s very likely the value of voting is terminal in some sense, based on cultural acclimation. Taking a broader view of why society itself developed this tendency, you might say that it did so for more consequentialist / instrumental reasons.

It’s similar to assessing the “reason” why a mother cares for her child. At an individual / neural level it’s based on reward circuitry. At a broader evolutionary level, it’s based on bequeathing genes.

Brian_Tomasik 13 Aug 2013 17:59 UTC
3 points
in reply to: Nick_Beckstead’s comment on: Common sense as a prior
1.3 billion Muslims—including many national leaders, scientists, and other impressive people throughout the Islamic world—believe that Mohammad was an authority on whether Allah exists.

Of course, there are plenty of others who do not, so I’m not suggesting we conclude it’s probable that he was, but it seems like something we couldn’t rule out, either.