It also seems worth noting that this study looked at whether people intervened in aggressive public conflicts, which is a type of situation where the bystander’s safety could be at risk and there can be safety in numbers. A lone bystander intervening in a fight is at higher risk of getting hurt, compared to a group of 10 bystanders acting together. This factor doesn’t exist (or is much weaker) in situations like “does anyone stop to see if the person lying on the ground needs medical help” or “does anyone notify the authorities about the smoke which might indicate a fire emergency.” So I’d be cautious about generalizing to those sorts of situations.
Unnamed
Karma: 5,441
The standard claim in bystander effect research is that an individual bystander’s probability of intervening goes down as the number of bystanders increases (see, e.g., Wikipedia). Whereas this study looked at the probability of any intervention from the group of bystanders, which is a different thing.
The abstract of the paper actually begins with this distinction:
Half a century of research on bystander behavior concludes that individuals are less likely to intervene during an emergency when in the presence of others than when alone. By contrast, little is known regarding the aggregated likelihood that at least someone present at an emergency will do something to help.
So: not a debunking. And another example of why it’s good practice to check the paper in question (or at least its abstract) and the Wikipedia article(s) on the topic rather than believing news headlines.
One angle for thinking about why the tails come apart (which seems worth highlighting even more than it was highlighted in the OP) is that the farther out you go in the tail on some variable, the smaller the set of people you’re dealing with.
Which is better, the best basketball team that you can put together from people born in Pennsylvania or the best basketball team that you can put together from people born in Delaware? Probably the Pennsylvania team, since there are about 13x as many people in that state so you get to draw from a larger pool. If there were no other relevant differences between the states then you’d expect 13 of the best 14 players to be Pennsylvanians, and probably the two neighboring states are similar enough so that Delaware can’t overcome that population gap.
Now, imagine you’re picking the best 10 basketball players from the 1,000 tallest basketball-aged Americans (20-34 year-olds), and you’re putting together another group consisting of the best 10 basketball players from the next 100,000 tallest basketball-aged Americans. Which is a better group of basketball players? In this case it’s not obvious—getting to pick from a pool of 100x as many people is an obvious advantage, but that height advantage could matter a lot too. That’s the tails coming apart—the very tallest don’t necessarily give you the very best basketball players, because “the very tallest” is a much smaller set than the “also really tall but not quite as tall”.
(I ran some numbers and estimate that the two teams are pretty similar in basketball ability. Which is a remarkable sign of how important height is for basketball—one pool has about a 4 inch height advantage on average, the other pool has 100x as many people, and those factors roughly balance out. If you want the example to more definitively show the tails coming apart, you can expand the larger pool by another factor of 30x and then they’ll clearly be better.)
Similarly, who has higher arm strength: the one person in our sample who has the highest grip strength, or the most arm-strong person out of the next ten people who rank 2-11 in grip strength? Grip strength is closely related to arm strength, but you get to pick the best from a 10x larger pool if you give up a little bit of grip strength. In the graph in the OP, the person who was 6th (or maybe 5th) in grip strength had the highest arm strength, so getting to pick from a pool of 10 was more important. (The average arm strength of the people ranked 2-11 in grip strength was lower than the arm strength of the #1 gripper, but we get to pick out the strongest arm of the ten rather than averaging them.)
So: the tails come apart because most of the people aren’t way out on the tail. And you usually won’t find the very best person at something if you’re looking in a tiny pool, even if that’s a pretty well selected pool.
Thrasymachus’s intuitive explanation covered this—having a smaller pool to pick from hurts because there are other variables that matter, and the smaller the pool the less you get to select for people who do well on those other variables. But his explanation highlighted the “other variables matter” part of this more than the pool size part of it, and both of these points of emphasis seem helpful for getting an intuitive grasp of the statistics in these types of situations, so I figured I’d add this comment.
And I said, in a move designed to be somewhat socially punishing: “I don’t really trust the conversation to go anywhere useful.” And then I took out my laptop and mostly stopped paying attention.
This ‘social punishment’ move seems problematic, in a way that isn’t highlighted in the rest of the post.
One issue: What are you punishing them for? It seems like the punishment is intended to enforce the norm that you wanted the group to have, which is a different kind of move than enforcing a norm that is already established. Enforcing existing norms is generally prosocial, but it’s more problematic if each person is trying to enforce the norms that he personally wishes the group to have.
A second thing worth highlighting is that this attempt at norm enforcement looked a lot like a norm violation (of norms against disengaging from a meeting). Sometimes “punishing others for violating norms” is a special case where it’s appropriate to do something which would otherwise be a norm violation, but that’s often a costly/risky way of doing things (especially when the norm you’re enforcing isn’t clearly established and so your actions are less legible).
When Scott used the term “asymmetric weapons”, I understood him to mean truth-asymmetric weapons or weapons that favor what’s good & true. He was trying to set that particular dimension of asymmetry apart from the various other ways in which a weapon might be more useful in some hands than in others.
I think it’s an important concept, and I wish we had better terminology for it.
Yes. Or at least, becoming a distinct organization was already the plan when I got there in early 2012: get a group of people together to create a rationality organization, initially rely on MIRI for institutional support, become an independent organization some months later once all the pieces are in place to do so.
Unilateral precommitment lets people win at “Almost Free Lunches”.
One way to model precommitment is as a sequential game: first player 1 chooses a number, then player 1 has the option of either showing that number to player 2 or keeping it hidden, then player 2 chooses a number. Optimal play is for player 1 to pick £1,000,000 and show that number, and then for player 2 to choose £999,999.99.
An interesting feature of this is that player 1′s precommitment helped player 2 even more than it helped player 1. Player 1 is “taking one for the team”, but still winning big. This distinguishes it from games like chicken, where precommitment is a threat that allows the precommitter to win the larger share. Though this means that if either player can precommit (rather than one being pre-assigned to go first as player 1) then they’d both prefer to have the other one be the precommitter.
This benefit of precommitment does not extend to the two option version (n2 vs. n1). In that version, player 2 is incentivized to say “n1” regardless of what player 1 commits to, so unilateral precommitment doesn’t help them avoid the Nash Equilibrium. As in the prisoner’s dilemma.
Some numbers related to c (how many capabilities researchers):
In 2018 about 8,500 people attended NeurIPS and about 4,000 people attended ICML. There are about 2,000 researchers who work at Google AI, and in December 2017 there were reports that about 700 total people work at DeepMind including about 400 with a PhD.
Turning this into a single estimate for “number of researchers” is tricky for the sorts of reasons that catherio gives. Capabilities researchers is a fuzzy category and it’s not clear to what extent people who are working on advancing the state of the art in general AI capabilities should include people who are primarily working on applications using the current art and people who are primarily working on advancing the state of the art in narrower subfields. Also obviously only some fraction of the relevant researchers attended those conferences or work at those companies.
I’ll suggest 10,000 people as a rough order-of-magnitude estimate. I’d be surprised if the number that came out of a more careful estimation process wasn’t within a factor of ten of that.
After discussing this offline, I think the main argument that I laid out does not hold up well in the case of blackmail (though it works better for many other kinds of threats). They key bit is here:
if Bob refuses and Alice carries out her threat then it is negative sum (Bob loses a lot and Alice loses something too)
This only looks at the effects on Alice and on Bob, as a simplification. But with blackmail “carrying out the threat” means telling other people information about Bob, and that is often useful for those other people. If Alice tells Casey something bad about Bob, that will often be bad for Bob but good for Casey. So it’s not obviously negative sum for the whole world.
There’s a pretty simple economic argument for why blackmail is bad: it involves a negative-sum threat rather than a positive-sum deal. I was surprised to not see this argument in the econbloggers’ discussion; good to see it come up here. To lay it out succinctly and separate from other arguments:
Ordinarily, when two people make a deal we can conclude that it’s win-win because both of them chose to make the deal rather than just not interacting with each other. By default Alice would just act on her own preferences and completely ignore Bob’s preferences, and the mirror image for Bob, but sometimes they find a deal where they each give up something in return for the other person doing something that they value even more. With some simplifying assumptions, the worst case scenario is that they don’t reach a deal and they both break even (compared to if they hadn’t interacted), and if they do reach a deal then they both wind up better off.
With a threat, Alice has an alternative course of action available which is somewhat worse for Alice than her default action but much worse for Bob, and Alice tells Bob that she will do the alternative action unless Bob does something for Alice. With some simplifying assumptions, if Bob agrees to give in then their interaction is zero-sum (Alice gets a transfer from Bob), if Bob refuses and Alice carries out her threat then it is negative sum (Bob loses a lot and Alice loses something too), and if Bob refuses and Alice backs down then it’s zero sum (both take the default action).
Ordinary deals add value to the world and threats subtract value from the world, and blackmail is a type of threat.
If we remove some simplifying assumptions (e.g. no transaction costs, one-shot interaction) then things get more complicated, but mostly in ways that make ordinary deals better and threats worse. In the long run deals bring people together as they seek more interactions which could lead to win-win deals, deals encourage people to invest in abilities that will make them more useful to other people so that they’ll have more/better opportunities to make deals, and the benefits of deals must outweigh the transaction costs & risks involved at least in expectation (otherwise people would just opt out of trying to make those deals). Whereas threats push people apart as they seek to avoid negative sum interactions, threats encourage people to invest in abilities that make them more able to harm other people, and transaction costs increase the badness of threats (turning zero sum interactions into negative sum) but don’t prevent those interactions unless they drive the threatmaker’s returns down far enough.
I think that there’s a spectrum between treating someone as a good source of conclusions and treating them as a good source of hypotheses.
I can have thoughts like “Carol looked closely into the topic and came away convinced that Y is true, so for now I’m going to act as if Y is probably true” if I take Carol to be a good source of conclusions.
Whereas if I took Alice to be a good source of hypotheses but not a good source of conclusions, then I would instead have thoughts like “Alice insists that Z is true, so Z seems like something that’s worth thinking about more.”
Giving someone epistemic tenure as a source of conclusions seems much more costly than giving them epistemic tenure as a source of hypotheses.
Huh? I am sufficiently surprised/confused by this example to want a citation.
Edit: The surprise/confusion was in reference to the pre-edit version of the above comment, and does not apply to the current edition.
I think we should take more care to separate the question of of whether AI developments will be decentralized with the question of whether decentralization is safer. It is not obvious to me whether a decentralized, economy-wide path to advanced AIs will be safer or riskier than a concentrated path within a single organization. It seems like the opening sentence of this question is carrying the assumption that decentralized is safer (“Robin Hanson has argued that those who believe AI Risk to be a primary concern for humanity, are suffering from a bias toward thinking that concentration of power is always more efficient than a decentralised system”).
I think you mean 50⁄62 = 0.81?
Sometimes theory can open up possibilities rather than closing them off. In these cases, once you have a theory that claims that X is important, then you can explore different values of X and do local hill-climbing. But before that it is difficult to explore by varying X, either because there are too many dimensions or because there is some subtlety in recognizing that X is a dimension and being able to vary its level.
This depends on being able to have and use a theory without believing it.
This sounds most similar to what LWers call generalizing from one example or the typical mind fallacy and to what psychologists call the false-consensus effect or egocentric bias.
Here are relatively brief responses on these 3 particular points; I’ve made a separate comment which lays out my thinking on metrics like the Big 5 which provides some context for these responses.
We have continued to collect measures like the ones in the 2015 longitudinal study. We are mainly analyzing them in large batches, rather than workshop to workshop, because the sample size isn’t big enough to distinguish signal from noise for single workshops. One of the projects that I’m currently working on is an analysis of a couple years of these data.
The 2017 impact report was not intended as a comprehensive account of all of CFAR’s metrics, it was just focused on CFAR’s EA impact. So it looked at the data that were most directly related to CFAR alums’ impact on the world, and “on average alums have some increase in conscientiousness” seemed less relevant than the information that we did include. The first few paragraphs of the report say more about this.
I’m curious why you’re especially interested in Raven’s Progressive Matrices. I haven’t looked closely at the literature on it, but my impression is that it’s one of many metrics which are loosely related to the thing that we mean by “rationality.” It has the methodological advantage of being a performance score rather than self-report (though this is partially offset by the possibility of practice effects and effort effects). The big disadvantage is the one that Kaj pointed to: it seems to track relatively stable aspects of a person’s thinking skills, and might not change much even if a person made large improvements. For instance, I could imagine a person developing MacGyver-level problem-solving ability while having little or no change in their Raven’s score.
Here’s a sketch of my thinking about the usefulness of metrics like the Big 5 for what CFAR is trying to do.
It would be convenient if there was a definitive measure of a person’s rationality which closely matched what we mean by the term and was highly sensitive to changes. But as far as I can tell there isn’t one, and there isn’t likely to be one anytime soon. So we rely on a mix of indicators, including some that are more like systematic metrics, some that are more like individuals’ subjective impressions, and some that are in between.
I think of the established psychology metrics (Big 5, life satisfaction, general self-efficacy, etc.) as primarily providing a sanity check on whether the workshop is doing something, along with a very very rough picture of some of what it is doing. They are quantitative measures that don’t rely on staff members’ subjective impressions of participants, they have been validated (at least to some extent) in existing psychology research, and they seem at least loosely related to the effects that CFAR hopes to have. And, compared to other ways of evaluating CFAR’s impact on individuals, they’re relatively easy for an outsider to make sense of.
A major limitation of these established psychology metrics is that they haven’t been that helpful as feedback loops. One of the main purposes of a metric is to provide input into CFAR’s day-to-day and workshop-to-workshop efforts to develop better techniques and refine the workshop. That is hard to do with metrics like the ones in the longitudinal study, because of a combination of a few factors:
The results aren’t available until several months after the workshop, which would make for very slow feedback loops and iteration.
The results are too noisy to tell if changes from one workshop to the next are just random variation. It takes several workshops worth of data to get a clear signal on most of the metrics.
These metrics are only loosely related to what we care about. If a change to the workshop leads to larger increases in conscientiousness that does not necessarily mean that we want to do it, and when a curriculum developer is working on a class they are generally not that interested in these particular metrics.
These metrics are relatively general/coarse indicators of the effect of the workshop as a whole, not tied to particular inputs. So (for example) if we make some changes to the TAPs class and want to see if the new version of the class works better or worse, there isn’t a metric that isolates the effects of the TAPs class from the rest of the workshop.
(This is Dan from CFAR)
CFAR’s 2015 Longitudinal Study measured the Big 5 and some other standard psychology metrics. It did find changes including decreased neuroticism and increased conscientiousness.
Number of weakly connected digraphs with n nodes.
http://oeis.org/A003027