Moloch’s Toolbox (2/2)
Follow-up to: Moloch’s Toolbox (1/2)
vii. Sticky traditions in belief-dependent Nash equilibria without common knowledge
cecie: I could talk next about a tax system that makes it cheaper for corporations to pay for care instead of patients, and how that sets up a host of “decisionmaker is not the beneficiary” problems.
But I suspect a lot of people reading this conversation understand that part already, so instead I’ll turn my attention to venture capital.
visitor: It sounds like the “politicians” and the “voters” might be a more key issue, if the cultural translator is right about what those correspond to.
cecie: Ah! But it turns out that venture capitalists and startups can be seen as a simpler version of voters and politicians, so it’s better to consider entrepreneurs first.
Besides, at this point I imagine the Visitor is wondering, “Why can’t anyone make any money by saving those babies? Doesn’t your society have a profit incentive that fixes this?”
visitor: Actually, I don’t think that was high on my list of questions. It’s understood among my people that not every problem is one you can make a profit by fixing—persistent societal problems tend to be ones that don’t have easily capturable profits corresponding to their solution.
I mean, yes, if this was all happening on our world and it wasn’t already being addressed by the Serious People, then somebody would just mix the bleeping nutrients and sell it to the bleeping parents for bleeping money. But at this point I’ve already guessed that’s going to be illegal, or saving babies using money is going to be associated with the wrong Tower and therefore unprestigious, or your parents are using a particular kind of statistical analysis that requires baby sacrifices, or whatever.
cecie: Hey, details matter!
visitor: (in sad reflection) Do they? Do they really? Isn’t there some point where you just admit you can’t stop killing babies and it doesn’t really matter why?
cecie: No. You can never say that if you want to go on being a cynical economist.
Now, there are several different kinds of molasses covering the world of startups and venture capital. It’s the tradition-bound aspects of that ecosystem that we’ll find especially interesting, since according to its own ideology, venture capitalists are supposed to chase strange new ideas that other venture capitalists don’t believe in. Walking through the simpler case of venture capital will help us understand the more complex reasons why voters and politicians are nailed into their own equilibria, underpinning the ultimate reasons why nobody can change the laws that prevent change.
visitor: (gazing off into the distance) … I wonder if maybe there are some worlds that can’t be saved.
cecie: Suppose it’s widely believed that the most successful entrepreneurs have red hair. If you’re an unusually smart venture capital company that realizes that, a priori, hair color doesn’t seem like it should correlate to entrepreneurial ability, you might think you could make an excess profit by finding some overlooked entrepreneur with blonde hair.
The key insight here is that venture capital is a multi-stage process. There’s the initial or pre-seed round, the seed round, the Series A, the Series B, the middle rounds, the Series C… and if the startup fails to raise money on any of those rounds before they become durably profitable, they’re dead. What this means is that the seed-round investors need to consider the probability that the company can successfully raise a Series A. If the angels invest in the seed round of a company whose entrepreneurs don’t have red hair, that company won’t be able to raise a Series A and will go bust and the angel investment will be worthless. So the angel investors need to decide where to invest, and what price to offer, based partially on their beliefs about what most Series A investors believe.
simplicio: Ah, I’ve heard of this. It’s called a Keynesian beauty contest, where everyone tries to pick the contestant they expect everyone else to pick. A parable illustrating the massive, pointless circularity of the paper game called the stock market, where there’s no objective except to buy the pieces of paper you’ll think other people will want to buy.
cecie: No, there are real returns on stocks—usually in the forms of buybacks and acquisitions, nowadays, since dividends are tax-disadvantaged. If the stock market has the nature of a self-fulfilling prophecy, it’s only to the extent that high stock prices directly benefit companies, by letting the company get more capital or issue bonds at lower interest. If not for the direct effect that stock prices had on company welfare, it wouldn’t matter at all to a 10-year investor what other investors believe today. If stock prices had zero effect on company welfare, you’d be happy to buy the stock that nobody else believed in, and wait for that company to have real revenues and retained assets that everyone else could see 10 years later.
simplicio: But nobody invests on a 10-year horizon! Even pension companies invest to manage the pension manager’s bonus this year!
visitor: Surely the recursive argument is obvious? If most managers invest with 1-year lookahead, a smarter manager can make a profit in 1 year by investing with a 2-year lookahead, and can continue to extract value until there’s no predictable change from 2-year prices to 1-year prices.
cecie: In the entrepreneurial world, startups are killed outright, very quickly, by the equivalent of low stock prices. And for legal reasons there are no hedge funds that can adjust market prices en masse, so the recursive argument doesn’t apply. The upshot is that seed investors have a strong incentive to care about what Series A investors think. If the entrepreneurs don’t fit the stereotype of cool entrepreneurs who have red hair, you can’t make an excess return by going against the popular misapprehension, because the startup will die in the next funding round.
The key phenomenon underlying the social molasses is that there’s a self-reinforcing equilibrium of beliefs. Maybe a lot of the Series A investors think the idea of entrepreneurs needing to have red hair is objectively silly. But they expect Series B investors to believe it. So the Series A investors don’t invest in blonde-haired entrepreneurs. So the seed investors are right to believe that “Series A investors won’t invest in blonde-haired companies” even if a lot of the reason why Series A investors aren’t investing is not that they believe the stereotype but that they believe that Series B investors believe the stereotype. And from the outside, of course, all that investors can see is that most investors aren’t investing in blonde-haired entrepreneurs—which just goes to reinforce everyone’s belief that everyone else believes that red-haired entrepreneurs do better.10
visitor: And you can’t just have everyone say those exact words aloud, in unison, and simultaneously wake up from the dream?
simplicio: I’m afraid people don’t understand recursion as well as that would require.
cecie: Perhaps, Simplicio, it is only that most VCs believe that most other VCs don’t understand recursion; that would have much the same effect in practice.
simplicio: Or maybe most people are too stupid to understand recursion. Is that something you’d be able to accept, if it were true?
cecie: Regardless, on a larger scale, what we’re seeing is an extra stickiness that results when the incentive to try an innovation requires you to believe that other people will believe the innovation will work. An equilibrium like that can be much stickier than a scenario where, if you believe that a project will succeed, you have an incentive to try it even if other people expect the project to fail.
Stereotypically, the startup world is supposed to consist of heroes producing an excess return by pursuing ideas that nobody else believes in. In reality, the multi-stage nature of venture capital makes it very easy for the field to end up pinned to traditions about whether entrepreneurs ought to have red hair—not because everyone believes it, but because everyone believes that everyone believes it.
viii. First-past-the-post and wasted votes
visitor: Does this feed back into our primary question of why your society can’t stop itself from feeding poisonous substances to babies?
cecie: It’s true that venture capitalists are now collectively skeptical of attempts at new drug development, but the real problem (at least for cases like this) is the enormous cost of approval and the long delays the FDA causes.11 The actual reason I went into this is that by understanding venture capitalists and entrepreneurs, we can understand the more complex case of voters and politicians. Which is the key to the political equilibrium that pins down the FDA, and all the other laws that prevent anyone from doing better. Not always, but quite often, the ultimate foundations of failure trace back to the molasses covering voters and politicians.
simplicio: I’d like to offer, throughout whatever theory follows, the alternative hypothesis that voters are in fact just fools, sheep, and knaves. I mean, you should at least be considering that possibility.
cecie: The simplest way of understanding the analogy between venture capitalists and voters is that voters have to vote for politicians that are electable.
visitor: Uh, what? When you write down your preference ordering on elected representatives, you need to put politicians that other voters prefer at the top of your preference ordering?
cecie: Yes, that’s pretty much what it amounts to. In the US, at least, elections are run on what’s known as a “first-past-the-post” voting system. Whoever gets the most votes in the contest wins. People who study voting systems widely agree that first-past-the-post is among the worst voting systems—it’s provably impossible for one voting system to have all the intuitively good properties at once, but FPTP is one of the most broken.
visitor: Why not vote to change the voting system, then?
cecie: I’ll get to that!
There are several ways of explaining what’s wrong with FPTP, but a lovely explanation I recently encountered phrases the explanation in terms of “wasted votes”—the total number of votes that can be removed without changing the outcome.
The two classic forms of gerrymandering are cracking and packing. Let’s say the parties are Green and Orange, and the Green party is in charge of drawing the voting boundaries. As a Green, you want to draw up districts such that Green politicians win with 55% of the vote—with some room for error, but not all that much—and for Orange politicians to win with 100% of the vote.
simplicio: Ah, so that the Orange politicians won’t need to be responsive to Orange voters because their re-election is nearly guaranteed, right?
cecie: No, the plot is far more diabolical than that. Consider a district of 100,000 people, where a Green politician wins with 55% of the vote. When 50,001 Green voters had cast their ballots, the election was already decided, under first-past-the-post, so the next 4,999 Green votes are “wasted”—this is to be understood as a technical term, not a moral judgment—in that they don’t further change the outcome. Then 45,000 Orange votes are also “wasted,” in that they don’t change the outcome. And also, one notes, those Orange voters don’t get the representative they wanted.
In an Orange district of 100,000 where the politician wins with 100% of the vote, there are 50,000 potent Orange votes and 50,000 wasted Orange votes. In total, there are 50,000 potent Green votes, 5,000 wasted Green votes, 50,000 potent Orange votes, and 95,000 wasted Orange votes. On a larger scale, this means that you can control a majority of a state legislature with slightly more than 1⁄4 of the votes—just have 55% of the districts containing 55% Green voters, with everything else solid Orange.
visitor: And then this quarter of the population rules cruelly over the remaining three-quarters, who in turn lack the weapons to rise up?
cecie: No, the real damage is far subtler. Let’s say that Alice, Bob, and Carol have taken time off from their cryptographic shenanigans to run for political office. Alice is in the lead, followed by Bob and then by Carol. Suppose Dennis prefers Carol to Bob, and Bob to Alice. But Dennis can’t actually write “Carol > Bob > Alice” on a slip of paper that gets processed by a trivially more sophisticated voting system. Dennis is only allowed to write down one candidate’s name, and that’s his vote. Under a system where the candidate with the most votes wins, and there’s uncertainty about which of the two frontrunners might win, all votes for whoever is in third place will be wasted votes, and this fact is predictable to the voters.
visitor: Ah, I see. That’s why you introduced your peculiar multi-stage system of venture capital, which I assume must be held in place by laws forbidding anyone else to go off and organize their own financial system differently, and observed how it creates a sticky equilibrium in which financiers must believe that other financiers will believe in a startup.
If Dennis doesn’t believe that other “voters” will believe in Carol, Dennis will vote for Bob, which makes your politics stickier than a system in which “voters” were permitted to support the people they actually liked.
cecie: Well, you see the analogy, but I’m not sure you appreciate the true depth of the horror.
visitor: I’m sure I don’t.
cecie: The upshot of first-past-the-post is typically a political system dominated by exactly two parties.
simplicio: Entities that tell sheep who to vote for.
cecie: In elections that have a single winner, votes for any candidate who isn’t one of the top two choices are wasted. In a representative democracy where districts vote on representatives who vote on laws, the dynamics of the district vote are then influenced by the dynamics of the national vote. Even if a third-party candidate could win a district, they wouldn’t have anyone to work with in the legislature, and so their votes would generally be wasted.
In the absence of a way to solve a large coordination problem, there’s no way for a third party to gain marginal influence over time. Each individual who considers voting for a third-party candidate knows they’ll be wasting their vote. This also means that third parties can’t field good candidates, since potential candidates know they’d be running to lose, which is stressful and unrewarding for people with better life options. And that’s a sufficient multi-factor system to prevent strong third parties from arising. When you’re not allowed to vote for Carol, who you actually like, you’ll vote for whichever of Alice and Bob you dislike the least.
The resulting equilibrium… well, Abramowitz and Webster found that what mainly predicted voting behavior wasn’t how much the voter liked their preferred party, but how much they disliked the opposing party.12 Essentially, the US has two major voting factions, “people who hate Red politicians” and “people who hate Blue politicians.” When the Red politicians do something that Red-haters really dislike, that gives the Blue politicians more leeway to do additional things that Red-haters mildly dislike, which can give the Red politicians more leeway of their own, and so the whole thing slides sideways.
simplicio: Looking at the abstract of that Abramowitz and Webster paper, isn’t one of their major findings that this type of hate-based polarization has increased a great deal over the last twenty years?
cecie: Well, yes. I don’t claim to know exactly why that happened, but I suspect the Internet had something to do with it.
In the US, the current two parties froze into place in the early twentieth century—before then, there was sometimes turnover (or threatened turnover). I suspect that the spread of radio broadcasting had something to do with the freeze. If you imagine a country in the pre-telegraph days, then it might be possible for third-party candidates to take hold in one state, then in nearby states, and so a global change starts from a local nucleus. A national radio system makes politics less local.
The Internet might have pushed this phenomenon further and caused most of politics to be about the same national issues, which in turn reinforces the Red-vs.-Blue dynamic that allows each party to sustain itself on hatred for the other.
But that’s just me trying to eyeball the phenomenon using American history—I haven’t studied it. Other countries that also have the radio and Internet and similar electoral dynamics do manage to have more than two relevant parties, possibly because of dynamics that cause the votes of third-party politicians to be less wasted.
simplicio: Isn’t the solution here obvious, though? All of these problems are caused by voters’ willingness to compromise on their principles and accept the lesser of two evils.
cecie: Would things be better if people chose the greater of two evils? If they acted ineffectually against that greater evil? The Nash equilibrium isn’t an illusion. Individuals would do worse by playing away from that Nash equilibrium. Wasted votes are wasted. The current system is an effective trap and the voters are trapped. They can’t just wish their way out of that trap.
There doesn’t need to be any way for good to win; and if there isn’t, the lesser evil really is the best that voters can do. Pretending otherwise may feel righteous, but it doesn’t change the equilibrium.
visitor: Just one second. Isn’t this all window dressing, compared to the issue of whatever true ruler imposes these rules on the “voters”? Like, if you put me into an elaborate cage that gives me an electric shock each time I vote for Carol, obviously the person who really controls the system is whoever put the cage in place and determines which politicians you can vote for without electric shocks.
simplicio: I like the way you think.
cecie: It’s not quite true to say that the system is self-reinforcing and that the voters are the sole instrument of their own destruction. But the lack of any obvious, individual tyrant who personally decides who you’re allowed to vote for has indeed caused many voters to believe that they are in control. I mean, they don’t feel like they’re in control, but they think that “the voters” select politicians.
They aren’t able to personalize a complicated bad equilibrium as a tyrant—not like they would blame a jeweled king who was standing in the polling booth, ready to give them an electric shock if they wrote down Carol’s name.
Inspired by Allan Ginsberg’s poem Moloch, Scott Alexander once wrote of coordination failures:
Moloch is introduced as the answer to a question—C. S. Lewis’ question in Hierarchy Of Philosophers—what does it? Earth could be fair, and all men glad and wise. Instead we have prisons, smokestacks, asylums. What sphinx of cement and aluminum breaks open their skulls and eats up their imagination?
And Ginsberg answers: Moloch does it.
There’s a passage in the Principia Discordia where Malaclypse complains to the Goddess about the evils of human society. “Everyone is hurting each other, the planet is rampant with injustices, whole societies plunder groups of their own people, mothers imprison sons, children perish while brothers war.”
The Goddess answers: “What is the matter with that, if it’s what you want to do?”
Malaclypse: “But nobody wants it! Everybody hates it!”
Goddess: “Oh. Well, then stop.”
The implicit question is—if everyone hates the current system, who perpetuates it? And Ginsberg answers: “Moloch.” It’s powerful not because it’s correct—nobody literally thinks an ancient Carthaginian demon causes everything—but because thinking of the system as an agent throws into relief the degree to which the system isn’t an agent.13
Scott Alexander saw the face of the Enemy, and he gave it a name—thinking that perhaps that would help.
visitor: So if you did do this to yourselves, all by yourselves with no external empire to prevent you from doing anything differently by force of arms, then why can’t you just vote to change the voting rules? No, never mind “voting”—why can’t you all just get together and change everything, period?
cecie: It’s true that concepts like these are nontrivial to understand.
It’s not obvious to me that people couldn’t possibly understand them, if somebody worked for a while on creating diagrams and videos.
But the bigger problem is that people wouldn’t know they could trust the diagrams and videos. I suspect some of the dynamics in entrepreneur-land are there because many venture capitalists run into entrepreneurs that are smarter than them, but who still have bad startups. A venture capitalist who believes clever-sounding arguments will soon be talked into wasting a lot of money. So venture capitalists learn to distrust clever-sounding arguments because they can’t distinguish lies from truth, when they’re up against entrepreneurs who are smarter than them.
Similarly, the average politician is smarter than the average voter, so by now most voters are just accustomed to a haze of plausible-sounding arguments. It’s not that you can’t possibly explain a Nash equilibrium. It’s that there are too many people advocating changes in the system for their own reasons, who could also draw diagrams that sounded equally convincing to someone who didn’t already understand Nash equilibria. Any talk of systemic change on this level would just be lost in a haze of equally plausible-sounding-to-the-average-voter blogs, talking about how quantitative easing will cause hyperinflation.
visitor: Maybe it’s naive of me… but I can’t help but think… that surely there must be some breaking point in this system you describe, of voting for the less bad of two awful people, where the candidates just get worse and worse over time. At some point, shouldn’t this be trumped by the “voters” just getting completely fed up? A spontaneous equilibrium-breaking, where they just didn’t vote for either of the standard lizards no matter what?
cecie: Perhaps so! But my own cynicism can’t help but suspect that this “trumping” phenomenon of which you speak would be even worse.
simplicio: I have a technical objection to your ascribing all these sins to first-past-the-post voting rather than, say, the personal vices of the voters. There are numerous parliamentary democracies outside the United States that practice proportional representation, where a party getting 30% of the votes gets 30% of the seats in parliament. And they don’t seem to have solved these problems.
cecie: Omegaven does happen to be approved in Europe, however. Like, they are not in fact killing those particular babies—
simplicio: Oh, come on! Yes, the European equivalent of the US’s FDA happens to be a bit less stupid. Lots of other things in European countries happen to be more stupid. Indeed, I’d say that in Europe you have much crazier people getting seats in parliaments, compared to the United States. The problem isn’t the voting system. The problem is the voters.
cecie: There are indeed some voters who want stupid things, and under the European system, their voice can be heard. There are also voters who want smart things and whose voices can be heard, like in the Pirate Party in Finland. But European parliamentary systems have different problems stemming from different systemic flaws.
Proportional representation would be a good system for a legislature that needed to repeatedly vote on laws, where different legislators could form different coalitions for each vote. If instead you demand that a majority coalition “form a government” to appoint an executive, then you need to give concessions to some factions, while other factions get frozen out. I’m not necessarily saying that it would be easy to fix all the problems simultaneously. Still, I imagine that a proportionally represented legislature, combined with an executive elected at-large by Condorcet voting, might possibly be less stupid—
simplicio: Or maybe it would just give stupid voters a louder voice. I don’t like the evil conspiracy of the press and political elites that governs my country from the shadows, but I am willing to consider the proposition that the alternative is Donald Trump. I mean, I intend to go on fighting the Conspiracy about many specific issues. But if you’re proposing a reform that puts more power into the hands of sheep not yet awakened, the results could be even worse.
cecie: Well, I agree that the design of well-functioning political systems is hard. Singapore might be the best-governed country in the world, and their history is approximately, “Lee Kuan Yew gained very strong individual power over a small country, and unlike the hundreds of times in the history of Earth when that went horribly wrong, Lee Kuan Yew happened to know some economics.” But the Visitor asked me why we were killing babies, and I tried to answer in terms of the system that obtained in the part of the world that was actually killing those babies. You asked why Europe wasn’t a paradise since it used proportional representation, and my answer is that parliamentary systems have their own design flaws that induce a different kind of dysfunction.
simplicio: Then if both systems are bad, how does your hypothesis have any observable consequences?
cecie: Because different systems are bad in different ways. When you have a “crazy” new idea, whether it’s good or bad, the European parliaments will be allowed to talk about it first. Whether that’s Omegaven, basic income, gay marriage, legalized prostitution, ending the war on drugs, land value taxes, or fascist nationalism, you are more likely to find it talked about in systems of proportional representation. It also happens to be true that those governments bloat up faster because of the repeated bribes required to hold the “governing coalition” together, but that’s a different problem.
ix. The Overton window
simplicio: I’m beginning to experience the same sort of confusion as the Visitor about your view of the world, Conventional Cynical Economist. If voters weren’t stupid, the world would look very different than it does.
If the ultimate source of stupidity were poorly designed governmental structures, then average voters would sound smarter than average politicians. I don’t think that’s actually true.
cecie: There are deeper forms of psychological molasses that generalize beyond first-past-the-post political candidates. The still greater force locking bad political systems into place is an equilibrium of silence about policies that aren’t “serious.”
A journalist thinks that a candidate who talks about ending the War on Drugs isn’t a “serious candidate.” And the newspaper won’t cover that candidate because the newspaper itself wants to look serious… or they think voters won’t be interested because everyone knows that candidate can’t win, or something? Maybe in a US-style system, only contrarians and other people who lack the social skill of getting along with the System are voting for Carol, so Carol is uncool the same way Velcro is uncool and so are all her policies and ideas? I’m not sure exactly what the journalists are thinking subjectively, since I’m not a journalist. But if an existing politician talks about a policy outside of what journalists think is appealing to voters, the journalists think the politician has committed a gaffe, and they write about this sports blunder by the politician, and the actual voters take their cues from that. So no politician talks about things that a journalist believes it would be a blunder for a politician to talk about. The space of what it isn’t a “blunder” for a politician to talk about is conventionally termed the “Overton window.”
simplicio: It’s all well and good to talk about complicated clever things, Cynical Economist, but what explanatory power does all this added complexity have? Why postulate politicians who believe that journalists believe that voters won’t take something seriously? Why not just say that people are sheep?
cecie: To name a recent example from the United States, it explains how, one year, gay marriage is this taboo topic, and then all of a sudden there’s a huge upswing in everyone being allowed to talk about it for the first time and shortly afterwards it’s a done deal. If you suppose that a huge number of people really did hate gay marriage deep down, or that all the politicians mouthing off about the sanctity of marriage were engaged in a dark conspiracy, then why the sudden change?
With my more complicated model, we can say, “An increasing number of people over time thought that gay marriage was pretty much okay. But while that group didn’t have a majority, journalists modeled a gay marriage endorsement as a ‘gaffe’ or ‘unelectable’, something they’d write about in the sports-coverage overtone of a blunder by the other team—”
simplicio: Ah, so you say it was a conspiracy by evil journalists?
cecie: No! Those journalists weren’t consciously deciding the equilibrium. The journalists were writing “serious” articles, i.e., articles about Alice and Bob rather than Carol. The equilibrium consisted of the journalists writing sports coverage of elections, where everything is viewed through the lens of a zero-sum competition for votes between Alice’s team and Bob’s team. Viewed through that lens, the journalists thought a gay marriage endorsement would be a blunder. And if you do something that enough journalists think is a political blunder, it is a political blunder. The journalists’ sports coverage will describe you as an incompetent politician, and primates instinctively want to ally with likely winners. Which meant the equilibrium could have a sharp tipover point, without most of the actual population changing their minds sharply about gay marriage in that particular year. The support level went over a threshold where somebody tested the waters and got away with it, and journalists began to suspect it wasn’t a political blunder to support gay marriage, which let more politicians speak and get away with it, and then the change of belief about what was inside the Overton window snowballed. I think that’s what we saw.
simplicio: Forgive me for resorting to Occam’s Razor, but is it not simpler just to say that people’s beliefs changed slowly until it reached some level where the military-industrial complex realized they couldn’t win the battle to suppress gay marriage outright, and so stopped fighting?
cecie: In a sense, that’s not far off from what happened, except without the evil conspiracy part. We might or might not be approaching a similar tipover point about ending the War on Drugs—a long, slow, secular shift in opinion, followed by a sudden tipover point where journalists model politicians as being allowed to talk about it, which means that politicians can talk about it, and then a few years later everyone is acting like they always thought that way. At least, I hope that’s where the current trend is leading.
simplicio: Several states have already passed laws legalizing marijuana. Why hasn’t that already broken the Overton window?
cecie: Because voter initiatives don’t break the common belief about what it would be a “gaffe” for a serious, national-level politician to do.
eliezer: (aside) What broke the silence about artificial general intelligence (AGI) in 2014 wasn’t Stephen Hawking writing a careful, well-considered essay about how this was a real issue. The silence only broke when Elon Musk tweeted about Nick Bostrom’s Superintelligence, and then made an off-the-cuff remark about how AGI was “summoning the demon.”
Why did that heave a rock through the Overton window, when Stephen Hawking couldn’t? Because Stephen Hawking sounded like he was trying hard to appear sober and serious, which signals that this is a subject you have to be careful not to gaffe about. And then Elon Musk was like, “Whoa, look at that apocalypse over there!!” After which there was the equivalent of journalists trying to pile on, shouting, “A gaffe! A gaffe! A… gaffe?” and finding out that, in light of recent news stories about AI and in light of Elon Musk’s good reputation, people weren’t backing them up on that gaffe thing.
Similarly, to heave a rock through the Overton window on the War on Drugs, what you need is not state propositions (although those do help) or articles in The Economist. What you need is for some “serious” politician to say, “This is dumb,” and for the journalists to pile on shouting, “A gaffe! A gaffe… a gaffe?” But it’s a grave personal risk for a politician to test whether the public atmosphere has changed enough, and even if it worked, they’d capture very little of the human benefit for themselves.
visitor: So… if this is the key meta-level problem… then why can’t your civilization just consider and solve this entire problem on the meta level?
cecie: Oh, I’m afraid that this entire meta-problem isn’t the sort of thing the “leading candidates” Alice and Bob talk about, so the problem itself isn’t viewed as serious. That is, journalists won’t think it’s serious. Meta-problems in general—even problems as simple as first-past-the-post versus instant runoff for particular electoral districts—are issues outside the Overton window. So the leading candidates Alice and Bob won’t talk about organizational design reform, because it would be very damaging to their careers if they visibly focused their attention on issues that journalists don’t think of as “serious.”
visitor: Then perhaps the deeper question is, “Why does anyone listen to these ‘journalists’?” You keep attributing power to them, but you haven’t yet explained why they have that power under your equilibrium.
cecie: People believe that other people believe what’s in the newspapers.
Well, no, that’s too optimistic. A lot of people do believe what’s in the newspapers, so long as it isn’t about a topic regarding which they have any personal knowledge or expertise. The Gell-Mann Amnesia Effect is the term for how we read the paper about subjects we know about, and it’s talking about how wet streets cause rain; and then we turn to the story about international affairs or dieting, and for some reason assume it’s more accurate.
There’s some level on which most people prefer to talk and believe within the same mental world as other people. Nowadays a lot of people believe what they read on, say, Tumblr, and hardly look at The New York Times at all. But even then they still believe that other people believe what’s in The New York Times. That’s what gives The New York Times its special power over the collective consciousness, far out of proportion to their dwindling readership or the vanishing real trust that individuals from various walks of life have in them—what’s printed in The New York Times determines what people believe other people believe.
simplicio: Do you truly lay all the sins of humanity at the feet of all this weird recursion? Or is this just a sufficiently weird hypothesis that you find it more fun to think about than the alternatives?
cecie: I’m not sure I’m pointing in exactly the right direction, but I feel that I’m pointing in the general direction of something that’s truly important to the Visitor’s most basic question. The Visitor keeps asking why, in some sense, on some sufficiently general level, we can’t just snap out of it. And to put it in the sort of terms you yourself might want to use, Simplicio, if we’re looking for an explanation of why we can’t just snap out of it, then it might make sense to point to a bad Nash equilibrium covering our collective consciousness and discussion. I suspect that the recursion, the dependency on what people believe other people believe, has a lot to do with making that a sticky equilibrium a la venture capital.
eliezer: (aside) Returning to my day job: As of 2017, I pretty commonly hear from AI researchers who are worried about AGI safety, but who say that they don’t dare say anything like that aloud. You could see this as either a good sign or a very bad sign, depending on how pessimistic or optimistic you previously were about the adequacy of academic discussion.
simplicio: But then what, on your view, is the better way?
cecie: Again, I could pontificate about various ideas, but that’s a different and harder question than looking at the actual equilibrium that currently obtains and forces doctors to poison babies. There doesn’t have to be a better way.
x. Lower-hanging altruistic fruit and bigger problems
(The Visitor takes a deep breath. When the Visitor speaks again, it is louder.)
visitor: Then what about your <untranslatable 17>?
cecie: Sorry? That word didn’t come through.
visitor: What about everyone on your entire planet who could possibly care about babies dying?
So your medical specialists are borked. From the magic-tower analogy, I assume your systems of learning are borked, and that means most of the parents whose responsibility it is to protect the child are borked. Your politicians are borked. Your voters are borked. Your planet has no Serious People who could be trusted to try alternative shoe designs, let alone lead the way on any more complex coordination problem. Your prediction markets, I suppose, are somehow borked in a way that prevents anyone from making a profit by correcting inaccurate policy forecasts… maybe they forecast wrongly bad consequences to unpopular policies, which therefore never get implemented in a way that shows up the inaccurate prediction, since you don’t have any way to test things on a smaller scale? Your economists must somehow be borked—
cecie: It’s more that nobody ever listens to us. They pay us and then they don’t listen to us.
visitor: —and your financial system is borked so that nobody can make a profit on saving those babies or doing anything else useful. I’m not stupid. I’ve picked up on the pattern at this point.
But what about everyone else? There are seven billion people on your planet. How is it that none of them step up to save these babies from death and brain damage? How is your entire planet failing to solve this problem?
cecie: That… sounds like a weird question, to an Earth person.
visitor: Whatever your problems are, surely out of seven billion human beings there have to be some who could see the problems as you’ve laid them out, who could try to rally others to the cause of saving those babies, who could do whatever it took to save them!
Even if your system declares that saving babies is only the responsibility of “doctors” or “politicians” or whoever is the Someone Else whose Problem it is, there’s no law of physics that stops someone else from walking up to the problem and accepting responsibility for it. Out of seven billion people in your world, I can’t believe that literally all of them are incapable of gathering together some friends and starting things down the path to getting a little fish oil into a baby’s nutritional mixture!
eliezer: I think I’ll step in myself at this point. There’s one other very general conclusion we can draw from seeing this ever-growing heap of dead babies. We might say, “the inadequacy of the part implies the inadequacy of the whole”—as we’ve defined our terms, if a part of the system is inadequate in X lives saved for Y dollars, then the whole system is inadequate in X lives saved for Y dollars. Someone who is motivated and maximizing will first go after the biggest inadequacy anywhere that they think they can solve, and if they succeed, it pushes forward the adequacy frontier for the whole system. Thus, we can draw one other general conclusion from the observation that babies are still being fed soybean oil. We can conclude that everyone on the planet who is smart enough to understand this problem, and who cares about strangers’ lives, and who maximizes over their opportunities, must have something more important to do than getting started on solving it.
visitor: (aghast) More important than saving hundreds of babies per year from dying or suffering permanent brain damage?
eliezer: The observation stands: there must be, in fact, literally nobody on Earth who can read Wikipedia entries and understand that omega-6 and omega-3 fats are different micronutrients, who also cares and maximizes and can head up new projects, who thinks that saving a few hundred babies per year from death and permanent brain damage is the most important thing they could do with their lives.
visitor: So you’re implying…
eliezer: Well, mostly I’m implying that maximizing altruism is incredibly rare, especially when you also require sufficiently precise reasoning that you aren’t limited to cases where the large-scale, convincing study has already been done; and then we’re demanding the executive ability to start a new project on top of that. But yes, I’m also saying that here on Earth we have much more horrible problems to worry about.
cecie: We’ve just been walking through a handful of lay economic concepts here, the kind whose structure I can explain in a few thousand words. If you truly perceived the world through the eyes of a conventional cynical economist, then the horrors, the abominations, the low-hanging fruits you saw unpicked would annihilate your very soul.
eliezer: And then some of us have much, much more horrible problems to worry about. Problems that take more than reading Wikipedia entries to understand, so that the pool of potential solvers is even smaller. But even just considering this particular heap of dead babies, we know from observation that this part must be true: If you imagine everyone on Earth who fits the qualifications for the dead-baby problem—enough scientific literacy to understand relevant facts about metabolic pathways, and the caring, and the maximization, and enough scrappiness to be the first one who gets started on it, meeting in a conference room to divide up Earth’s most important problems, with the first subgroup taking on the most neglected problems demanding the most specialized background knowledge, and the second taking on the second-most-incomprehensible set of problems, until the crowdedness of the previously most urgent problem decreases the marginal impact of further contributions to the point where the next-worst problem at that level of background knowledge and insight becomes attractive… and so on down the ladders of urgency inside the levels of discernment… then there must be such a long and terrible list of tasks left undone, and so few people to understand and care, that saving a few hundred babies per year from dying or suffering permanent brain damage didn’t make the list. So it has been observed, and so it must be.
wandering bystander: (interjecting) But I just can’t believe our planet would be that dysfunctional. Therefore, by backward chaining, I question the original observation on which you founded your inference. In particular, I’m starting to wonder whether omega-3 and omega-6 could really be such significantly different micronutrients. Maybe that’s just a crackpot diet theory that somehow made it into Wikipedia, and actually all fats are pretty much the same, so there’s nothing especially terrifying about the prospect of feeding babies exclusively fat from soybean oil instead of something more closely resembling the lipid profile of breast milk?
eliezer: Ah, yes. I’m glad you spoke up. I’ll get to your modest proposal next.
Next: Living in an Inadequate World.
The full book will be available November 16th. You can go to equilibriabook.com to pre-order the book, or sign up for notifications about new chapters and other developments.
See Glenn Loury’s The Anatomy of Racial Inequality for an early discussion of this issue. Note that some venture capitalists I’ve spoken to endorse this as an account of VC dysfunction, while others have different hypotheses. ↩
Carl Shulman argues that the FDA’s clinical trial requirements probably aren’t the reason for recent decades’ slowdown in the development of cool new drugs, given that increased regulation seems to have coincided with but not substantially accelerated the declining efficiency of pharmaceutical research and development (source). Shulman suggests that Baumol’s cost disease and diminishing returns play a larger role in the R&D slowdown.
The FDA’s clinical trial requirements are much more likely to play a central role in limiting access to non-patented substances, though it’s worth noting here that the FDA has gotten faster than it used to be (source). ↩
Abramowitz and Webster, “All Politics is National.” ↩
See Scott Alexander’s “Meditations on Moloch.” ↩
- Politics is way too meta by 17 Mar 2021 7:04 UTC; 271 points) (
- The inordinately slow spread of good AGI conversations in ML by 21 Jun 2022 16:09 UTC; 173 points) (
- Moloch’s Toolbox (1/2) by 4 Nov 2017 21:46 UTC; 148 points) (
- Living in an Inadequate World by 9 Nov 2017 21:23 UTC; 87 points) (
- The inordinately slow spread of good AGI conversations in ML by 29 Jun 2022 4:02 UTC; 59 points) (EA Forum;
- Politics is far too meta by 17 Mar 2021 23:57 UTC; 58 points) (EA Forum;
- Charging for the Dharma by 11 Nov 2022 14:02 UTC; 29 points) (
- Adversarial epistemology by 24 Aug 2022 16:57 UTC; 25 points) (
- Trying for Five Minutes on AI Strategy by 17 Oct 2018 16:18 UTC; 18 points) (
- Moloch’s Toolbox (2/2) by 6 Nov 2017 21:34 UTC; 9 points) (EA Forum;
- 31 Jan 2020 19:26 UTC; 3 points)'s comment on REVISED: A drowning child is hard to find by (
Soooo.… why doesn’t someone build an app for this??
I mean, seriously. As Part 1 pointed out, we have Kickstarter, but Kickstarter only solves problems where it’s obvious that directly applying cash is what is needed. As soon as it gets more complicated than that, you need to trust the person who is going to be spending the cash, and then you get back into assymetric information land.
Let’s take the subset of problems where a) no one is afraid to be publically affiliated with the hard-to-coordinate action, they just can’t rationally take it without expecting everyone else to take it, and b) getting a sufficient number of people to agree to do it makes it rational.
A + B constrain the set of problems a lot. There are a lot of coordination problems where there are social and reputational costs to being outspoken prior to your side winning. But there are still plenty of important, hard problems where A + B apply. I think stuff around academia and the academic job market has a lot of things in this space. For instance, academics don’t seem to get penalized for publically saying “hey, wouldn’t it be great if we all stopped using p-values”, they only get penalized if they actually stop using them.
So, for this constrained subset of problems, what if there was an app that let you manage a campaign to coordinate, with increasing levels of escalation, from:
Expressing anonymous interest and being on the ping list
Expressing public interest
Expressing public commitment to take a specific action if X number of people also commit
Actually confirming (via taking a selfie or something?) that you’ve taken the action
The app helps you promote and manage a campaign such that people are only called upon to take action when there’s credible assurance that enough other people are taking it to make it successful.
I don’t see this is a panacea for all problems, but this certainly seems like it would knock out a large subset of messt ones. Anyone see any reason this wouldn’t be a great idea?
Worth noting: Eliezer failed to mention the phrase “assurance contract”, but that’s what the thing he was talking about is called. If you want to see what’s been done with assurance contract platforms already, it’s something you’ll want to look up.
Thanks! Always helpful to know what the actual term is. I did a couple minutes of googling… the one contemporary player I turned up is https://collaction.org. They seem to be playing from the modern web startup playbook (design aesthetic is Kickstarter-lite), but they don’t seem to have much traction: they claim six people on their team, but no evidence of revenue or fundraising, and their website is slow and a little clunky.
Their demo projects are pretty uninspiring; they don’t seem to be going after genuine collective action problems, but rather they’re just trying to see if they can get 50 or so people to commit to something: for instance “Random Act of Coffee: If 50 people pledge to buy an (extra) coffee for the next person to order, we will all do it!”
If I were trying to launch something like this, I think I would take on one project at a time, and pick something inspiring and ambitious enough that it might actually go viral, rather than try to get lots of small wins that aren’t really wins.
Getting people to use assurance contracts to solve coordination problems seems like another coordination problem. It would be funny to use them for that problem too. That is, get people to sign an assurance contract that they will start using more assurance contracts when enough others sign it ;)
To me this seems like a project of virtue signaling to each other. I don’t think the problem is that we have to little people who say “we should stop using p-values”. The problem is rather that we don’t have a good alternative on which we can agree of how the system should be structured.
Could you give specific examples of how you see specific problems getting solved?
Does anyone have a reference to a source or sources with more detail on what Eliezer is getting at here? I have a vague idea on the subject but a deeper dive would be illuminating.
Omegaven® is manufactured by German pharmaceutical company Fresenius Kabi. For some reason, the company decided to stay away from US market and this raised questions when announced back in 2006. Until patents held by FK are expired, no one in USA can sell Omegaven without license from FK.
Brief search in Clinical Trials registry gives 14 open clinical studies of Omegaven as Parenteral Nutrition in USA. I hope at least some of them don’t just pursue scientific goal of replicating earlier, but are compassionate attempts to provide an access to Omegaven by an Expanded Access Use program from FDA.
Several hundred saved children sadly is indeed too few for pharma to seriously care. The cost of clinical trial required for regulatory approval is ~ 100 M$ + about 100 M$ is required to set up manufacture, sales etc. With generous $ 100 000 per course of application (approx. a cost of life saved for pediatric anti-cancer drugs) and 200 patients/year, a company can generate $20 M revenue, so it has to wait 10 years just to cover the losses. And that without taking into account 30% IRR for VC, possible competitors undermining market share and so on.
Surely I can’t be the only one skeptical of Eliezer’s Gaff Theory? Indeed, journalists would often applaud policies with very little support if it fitted in with their political beliefs. Instead, I suspect that politicans don’t want to support a policy until it moves over that 50% threshold, at which point they switch en-mass to supporting it, not because of the media, but because they want votes.
I think the Gaffe Theory is approximately correct. My sense is that there are two Overton Windows, one for what serious candidates can say, and one for what a mainstream publication can print an op-ed about.
Any policy-centered theory like this runs into the problem that voters don’t vote based on policy. From a NY Times discussion of why Sanders voters in the 2016 primary generally supported Clinton’s policies more than Clinton voters did, and vice versa:
I think Cecie is right that press coverage of things like “gaffes” and “scandals” often focus intensely on the recursive issue of whether a gaffe will make someone “controversial” (unelectable, offensive, unpopular, etc.), neglecting or entirely skipping over the question of whether the gaffe was factually accurate. The last few times I’ve actually looked at news stories about controversial things, whether during elections or not, the coverage was very strongly and explicitly focused on perception over substance.
One way of putting it is that journalists care a lot about what the voters(/consumers/public) think, and this emphasis on electability over substance can then run away with itself. Another way of putting it is that voters care a lot about what other voters think. We’re all perpetually nervously glancing around to try to spot signs of what everyone else’s plans and preferences are, and what the new social norms and expectations are; journalists then pick up on this nervous curiosity and give the voting public what it wants, and also have an outsized impact on those norms and expectations because they’re an unusually centralized and concentrated source of evidence about norms/expectations.
Regardless of how this pattern develops, once it’s in place it tends to be self-reinforcing, because any additional coverage of “controversies” both establishes more firmly what things are versus aren’t controversial (including what’s inside the Overton window policy-wise), and establishes more firmly that it’s normal to focus on controversialness and what-other-people-are-thinking-about-this over substance.
Two places where it looks like I disagree with Cecie (and agree with Eliezer and possibly Simplicio) are that voters aren’t actually voting rationally, and in particular, voters aren’t just trying to model other voters for strategic reasons; they’re also emotionally invested in the idea of supporting strong, impressive, successful, high-status people. Gaffes don’t just provide evidence that candidates or policies will fail; they provide evidence that it will be embarrassing, low-status, or otherwise unpleasant to be perceived as supporting those candidates or policies. Even the minority who support underdogs want to feel like they’ve found a strong, impressive, successful, high-status underdog who’s got a real shot at victory against all odds, and loses only due to bias/unfairness and not due to social-status-lowering personal failings.
I think the idea that gaffes are serving as a proxy not only for acceptability-to-others but also for weakness (which in turn provides some evidence about emotional appeal-to-others) helps make a lot more sense of Donald Trump’s appeal. See also Eliezer on Gruber and on Trump.
To test whether this is an important factor, we might look at how well “losers” perform in non-FPTP systems. In FPTP, it makes sense to shun candidates based purely on the fact that you’re worried other people might shun them. In other systems, this makes less sense, so we can better isolate how much of the effect is about overactive primate coalition-building instincts versus sensible voting strategy.
I think you have described their real motivation, but how would they know? Politicians don’t have a secret database of polling data unavailable to the rest of us, or a deep expertise in reading the ones which are available.
The problem is both the voters and the politicians are drawing on the journalists for information. The journalists are writing for clicks, which means a national focus for the widest audience. The politicians then spend their time sounding good to their local constituents about these national trends, which is why they all sound insane to everyone else.
There is local variation in the perception of the national trends, which leads to everyone having a different conversation.
No, we live in the 21st century.
Politicians these days do pay pollsters to optimize their messaging. They let their pollsters run focus groups to find out what the public wants to hear.
Politicians raise money from lobbyists to be able to pay for their pollsters to know what to say, to get the public to like them. For better or worse it’s not the problem of US politics that politicians have to depend on the media to understand what the public thinks.
Regarding the multi-stage venture capital example: If only a small subset of VC companies recognize the value of a company due to a weird signalling equilibrium, and refuse to fund the valuable companies in successive rounds, that should make the market more exploitable, not less! It means they will have exclusive access to fund future rounds of the company if the company is successful. This argument seems to only fail if there is insufficient capital among the doubters to fund companies through to profitability (minding the variance involved). It seems like there is a ton of capital floating around, though, so I would be surprised if this was the case for anything but the most niche examples.
Am I wrong about this?
Elon Musk likely wouldn’t have been able to fund SpaceX or Tesla if he wouldn’t have been wealthy to begin with.
But the problem isn’t as bad as EY makes it out to be. Chamath Palihapitiya’s Social Capital hedge fund for example doesn’t have this problem as it’s willing to fund startups through all stages. Glooko might otherwise not have been successful.
YCombinator also tries to reduce this problem by instituting their growth fund.
To be complete, many series A investor don’t do series B financing is because this creates bad signals in the market for companies where the inve does the series A but doesn’t do the series B.
It’s right that people can self-fund the startup to completion. But notice that this necessarily reduces the value of the startup, because you are missing some option value (the ability to sell off pieces of it before IPO for additional capital for growth). And the opportunity cost for that money is likely buying into other startups.
If what really matters is having 5% of all the startups, say, rather than 100% of the 5% most promising startups, then having to self-fund them all the way is bad. (Especially so since money goes much further in the earlier rounds, meaning that if you have sharper eyes than other funds, a thing you’re hoping to do is buy the equity when it’s cheap and trust in the followers to buy the equity when it’s expensive.)
This is wrong. It’s overgeneralizing based on one example of US politics. If you look at the world it’s easy to see third parties getting influence. The UK parliament (with first-past-the-post) has 8 parties and additionally five independents.The liberal party become part of the government in 2010 and todays UK government has the conversative party needing the support of the small DUP. It gives them policy concessions in return.
Discussing what’s wrong with US politics without speaking about lobbyists and campaign finance seems strange. It’s much more important than the voting system. Robert Moses became the most powerful man in New York politics without ever winning a public election.
I’m voting in Berlin and in the last election we had a small party that called themselves the humanists. Their stated objective was to be evidence-based and pro-human rights. Wouldn’t it be good if all reasonable people would come together and fought for supporting this party?
No, it wouldn’t.
Our current medical system in Germany has different insurance companies and insurance companies are allowed to pay for some treatment they believe to be helpful even in the absence of two studies with statistical significant results. This is supposed to encourage them to be in competition with each other and do what’s optimal in terms of preventive medicine. The agenda of the humanist party is that this provides the companies with too much freedom and that we need stronger medical regulation.
There’s also a complete absence in the program of the party for ideas such as running more trials for public policy. The party just treats being evidence-based as an ideology and as long as you do the things that the New Atheist crowd considers to be evidence-based everything is fine.
It’s not enough to have a politican who is “pro-evidence-based”. Voting alone isn’t enough. You need actual public debate about the policies and build an understanding among experts who spend more time with a topic than the average citizen about what’s the best solution.
>CECIE: There are indeed some voters who want stupid things, and under the European system, their voice can be heard. There are also voters who want smart things and whose voices can be heard, like in the Pirate Party in Finland. But European parliamentary systems have different problems stemming from different systemic flaws.
The UK has first-past-the-post in the for electing parliamentarians just as the US has first-past-the-post. Blaming the problems on first-past-the-post doesn’t explain the difference.
Closer to the US you have Canada that has a first-past-the-post and has 5 political parties who have parliamentarians with in the House of Commons.
The idea that you don’t get more than two parties just because you have first-past-the-post isn’t based on good reasoning.
The US does have more than 2 parties though. I think the argument being made is less about there being only 2 parties and more to do with how power is distributed. In a FPTP system, you will really only have 2 major parties that swap power around, even if a third party can attract a significant number of votes. That is essentially the main strength of FPTP, it almost always produces a dominant victor. In practice the Canadian system has seen power swap between the Conservatives and Liberals. In the UK power has swapped between the Conservatives and Labour. In India, power has swapped between the BJP and the INC. In Mexico there have only ever been 2 parties. I’ve tried to find more examples but unfortunately most countries with FPTP do not have good wikipedia pages for electoral results.
The US does seem to have a uniquely dysfunctional system, but I don’t really see any significant evidence from other countries with FPTP of third parties being able to enact real change.
The UK just left the European union because of pressures from UKIP. Now, third parties have less seats than before but they did produce change.
Furthermore, parties in FPTP don’t vote in block in the same way they do in states with proportional representation. A parliamentarian in a FPTP can often affect law without having to be in the majority coalition. Pork is likely the best way to measure individual power. You don’t see pork that’s added to bills completely shift in the US congress and senate when the congressional or senate majorities change.
Indirectly, pressure from UKIP led to the current Brexit situation—which as gjm points out, has not yet resulted in the UK leaving. However, UKIP’s vote increase didn’t cause Brexit, it simply led to a referendum. But the conservatives could have easily not called the referendum in the end since UKIP’s high vote share did not translate into any seats in government. I think it’s much easier to pin the blame on Cameron’s arrogance and putting party politics ahead of country, than it is on UKIP.
Nonetheless, even if you do not agree with that assessment, Brexit remains one data point. I am not personally aware of any other such events occuring in the likes of Canada or India, or similar examples in the UK.
I know little of the US pork barrel system, so can’t offer comment on that.
The UK has not left the European Union. It has (narrowly) voted to do so, and it has issued the EU with formal notification of its intention to leave, and the obvious assumption is that it will in fact leave, but right now the UK is still in the EU and it’s probably not impossible for it to stay in if its politicians thought they would improve their chances of reelection by making that happen.
I agree that UKIP seems to have had a lot of influence despite being a small party. I do wonder whether a group with similar goals that wasn’t an actual political party could have had comparable influence, but on the whole I think probably not.
They don’t have FPTP for a broad national leadership, though, and that’s the important thing.
The Coalition government of 2010 was the first coalition in almost a century, and indeed you can find only 2 coalition governments in modern British political history. It was not the norm at all, and was heavily influenced by the bizarre regional nature of British politics. Meanwhile the most recent election can be seen as another FPTP aberration, since the most representative government would have been a Lab-Lib-SNP coalition rather than a Con minority. What is more, 2017 saw minor parties almost completely wiped out with 82% of votes and 89% of seats going to the 2 largest parties. The UK is a poor example of third party influence, I would say.
Speaking about “bizare regional nature” of politics basically means that there are political factors in play that are outside of your model for analysis politics.
If you have learned in school that political power flows through very specific formal channels than you will find informal ways in which political power flows bizare.
Wikipedia summarizes the insight that got Robert Caro to write the biography of Robert Moses as:
Telling people that American politics is so messy as it is because of formal arguments about first-past-the-post voting is similar to explaining that the way highways get build with formal mathematical formulas about traffic density.
Even if you would have ranked choice voting system that wouldn’t change anything substantial about the quality of political decisions. Yes, it might be a bit more optimal but it wouldn’t change anything about the underlying core problems.
A couple years ago, there were a bunch of sensational headlines along the lines of “US is an Oligarchy, not a Democracy, New Study Finds”. The actual study is an interesting read: Testing Theories of American Politics: Elites, Interest Groups, and Average Citizens
It correlated the average preferences of average citizens, economic elites, and special interest groups (divided into “mass public interest groups” and “business interest groups”). Interestingly, Table 2 shows that economic elite preferences actually correlate with average citizen preferences at .78. Also, business interest groups correlate with mass public interest group preferences at −0.05, but all interest groups overall at 0.96, so I guess the vast majority of interest groups are business interest groups? Aside from “all interest groups”, business interests seem to correlate negatively with everything else, although not as negatively as I would have thought.
They then looked at the probability of various legislation passing, given that various categories were for or against it on average. They focus on cases where, for example, economic elites favor something but average citizens are against it, and compare it to actual policy outcomes passing or failing. They build several models for power and influence, and do a bunch of statistics I haven’t read the details of to try to figure out how much influence various groups have on actual policy outcomes.
Eyeballing figure 1, if looks like if 10% of average americans support something, it has maybe a 30% chance of passing, but if 90% of average citizens support something, it has maybe a 31% chance of passing. Conversely, if 10% of economic elites favor a piece of legislation, it has only a 10% chance of passing, but if 90% of them favor it, it has a 60% chance of passing.
Both of those graphs were fairly linear, and had a good spread, with lots of legislation with both high support and high opposition. They also tried to graph the net number of interest groups in support or opposition to legislation, vs. the probability of it passing. However, in almost all cases, there were less than 4 more either supporting or opposing each bill, so all their data points are clustered in the center. I’m not sure I believed their model projections outside of that narrow range, but if you believe that weird-shaped s-curve, then more net interest groups in support of a bill does cause it to be much more likely to pass.
Take all this with a grain of salt, though. There have been a couple follow up studies which stepped this back substantially. As Vox puts it “When the rich and middle class disagree, each wins about half the time”:
That makes you wonder: If only 7.4% of the variance is due to these groups, then what on earth is determining policy??!!
Right. Obvious answer is obvious.
Or maybe the middle class, since they are conspicuously absent from “rich, the poor, and interest groups put together”? I think “rich” is still being defined as top 10% here, and “poor” as bottom 10%.
Or, some interest groups are much more effective than others, so their preferences will be the main driver of everything, but only weekly correlated with the average interest group preference. The same could be true of certain influential elites, if their opinions diverged from those of the rest of the elites. But, that’s a wild guess on my part. This whole thing is a mess I haven’t even begun to sort out in my head.
But it’s probably Moloch.
People like Robert Moses are neither “the rich” nor “the poor”. The same goes for various burocrats who have power inside of governments.
>Lee Kuan Yew gained very strong individual power over a small country, and unlike the hundreds of times in the history of Earth when that went horribly wrong, Lee Kuan Yew happened to know some economics.
Actually, this isn’t a one-off. Monarchies in general achieve superior economic results (https://twin.sci-hub.cc/6b4aea0cae94d2f4fd6c2e459dab6881/besley2017.pdf ):
>We assemble a unique dataset on leaders between 1874 and 2004 in which we classify them as hereditary leaders based on their family history. The core empirical finding is that economic growth is higher in polities with hereditary leaders but only if executive constraints are weak. Moreover, this holds across of a range of specifications. The finding is also mirrored in policy outcomes which affect growth. [...] The logic that we have exploited is essentially that put forward in Olson (1993) who emphasized that hereditary rule can provide a means improving inter-temporal incentives in government.
I’m not super convinced. Their non-empirical part seems tautological, and their empirical part seems like they kind of tried to separate between the subgroups of democracy and “junta leader of the month” by calling one “strong executive constraints” and the other “weak executive constraints,” but it’s not obvious that you’d expect this method of separation to work (though not that when they did split the data this way, the “democracy-esque” subgroup had the best results). Not only are natural experiments thin on the ground here, but it’s very hard to control for confounders, and I see no reason to expect they succeeded.
Also, they cited Thomas Paine as from 1976.
Heartbreaking :’( still, that “taken time off from their cryptographic shenanigans” line made me laugh so hard I woke my girlfriend up
The reason you can’t just vote to change the voting system actually seems a lot simpler to me, based on my experience of the UK AV referendum. The system actually worked well enough to get the idea of switching to a basically sane voting system put to a public referendum. Then look at how that referendum goes. Who stands to benefit?
The public (but they don’t know that)
The small political parties (who lack funding and are unable to mobilise large numbers of voters almost by definition)
Who stands to lose out
The major political parties (who have vast, well funded propaganda machines, good contacts, expertise etc)
So the red party told red voters that using a sane voting system would be bad for the red party, and the blue party told blue voters that using a sane voting system would be bad for the blue party, and the referendum came out as a ‘No’.
You can find endless examples of things that are some illogical way largely because everybody believes that everybody believes something. For instance, why don’t most buildings have 13th floors? It’s not likely to be because the builders are all superstitious in this way. Instead, they do it because they’re afraid that it would be harder to sell a building with a 13th floor in it, because prospective owners will be afraid it would be harder to rent or lease space on that floor, because prospective tenants/leasers will be afraid that prospective customers, employees, guests, visitors, etc. will be superstitious about it and feel uncomfortable going there. In this way, it’s entirely possible that even in a society where nobody actually is superstitious about the number 13 any more, everybody still persists in leaving it out of buildings. (So why do cities frequently have a 13th Street in numbered street systems? Maybe city planners’ incentives are different?)
I don’t buy the idea that voters are not the main source of problem and that its voting systems. Voters don’t have good incentives to have sensible opinions that go against natural prejudices.
It seems to me that if you found the median political opinion of people, you would have much worse policies in a lot of areas. Probably some would be better, but I would be surprised if it were many.
Two things: 1) A medium-sized correction, and 2) a clarification of something that wasn’t clear to me at first.
1) The correction (more of an expansion of a model to include a second-order effect) is on this bit:
It’s hard to see how 10-year time horizons could be common enough to overcome the self-fulfilling-prophecy effect in the entire stock market, when a third of all companies will be gone or taken over in 5 years. :p
We can edit the model to account for this in a couple ways. But this depends on whether investors are killing companies, and what the die off rate is for larger, fortune 500 companies. As I understand it, the big companies mostly aren’t optimizing for long term survival, but there are a few that are. I’d expect most to optimizing for expected revenue at the expense of gambler’s ruin, especially because the government subsidizes risk-taking by paying debts after bankruptcy.
(I’m not necessarily against bankruptcy though, as I understand that it makes companies much more willing to do business with each other, since they know they’ll get paid.)
I don’t know the details, but I’d lean a lot further toward the full Simplicio position than the chapter above does. That is, markets really are at least partly a Keynesian beauty contest / self-fulfilling prophecy.
2) Also, this section was confusing for me at first:
Let me make sure I understand, by stating the model explicitly: before effective communication, we were polarized locally but not so much nationally. You might have green and purple tribes in one state, and orange and yellow tribes in another. Now, as society is less regional, all these micro-tribalisms are aligning. I’m envisioning this as a magnetic field orienting thousands of tiny regions on a floppy disk, flipping them from randomized to aligned.
Ah, I understand what you were getting at with the leeway model now. To state the axioms it’s built from explicitly, coalitions form mainly based on a common outgroup. (Look at the Robbers Cave experiment. Look at studies showing that a shared dislikes builds friendships faster than a shared interests.)
So, if we vote mainly based on popularity contests involving which politician appears to have the highest tribal overlap with us, rather than policy, then the best way for politicians to signal that they are in our ingroup is to be offensive to our outgroup. that’s what leads to a signaling dynamic where politicians just get more and more hateful.
It’s not clear what the opposing forces are. There must be something, or we’d instantly race to the bottom over just a couple election cycles. Maybe politicians are just slow on adopting such a repulsive strategy? Maybe, as Eliezer suggests, it’s that politicians used to have to balance being maximally repulsive to lots of tiny local factions, but now are free to maximally repulsive to one of two single, fairly unified outgroups?
One relevant factor from the dialogue is that the Overton window limits politicians’ ability to creatively offend each other; “serious” policies and behaviors will tend to be relatively mainstream, traditional, and non-outrageous.
Thanks. The Overton Window stuff was mainly about why First Past The Post might be stuck in metaphorical molasses, and I hadn’t generalized the concept to other things yet.
Side note: this also gives an interesting glimpse into what it feels like from the inside to have one’s conceptual framework become more interconnected. Tools and mental models can exist happily side by side without interacting, even while explicitly wondering about a gap in one’s model that could be filled by another tool/model you already know.
It takes some activation energy (in the form of Actually Trying, ie thinking about it and only it for 5+ minutes by the clock), and then maybe you’ll get lucky enough to try the right couple pieces in the right geometry, and get model that that makes sense on reflection.
This suggests that re-reading the book later might be high-value, since it would help increase the cross-linking in my Bayesian net or whatever it is our brains think with.
For the same reasons as the first half of this chapter, I’ve promoted this to Featured.
I was quite surprised that nobody brought this up yet, but to me it is fairly clear that this is what Taleb’s antifragility MUST look like upon close inspection: a lot of “free energy” trapped in such a way that only destructive shocks can release it. Any system that actually benefits from destructive shocks must look mad, consisting of crazy parts. If all the trillion-dollar-bills lying around were accessible (and hence immediately picked up) “in peace time”, there would be no reserve for dealing with unexpected shocks from unknown unknowns well outside of the civilization’s current model of reality.
The Visitor’s civilization is fragile: there is no way a random destructive shock can improve on it, as if it could, they would have done that improvement already without the destructive shock. Destructive shocks can only harm it to various degrees. Whereas Earthlings might just stop the industrial-scale killing of babies (a huge win, obviously), if some important parts of our civilization get destroyed.
This sort of thing actually does happen. The reason North America currently uses 110-125V household AC is that when the standards were created about a century ago, the best insulators could safely insulate only for such voltages. Ever since North America is trapped in a Nash equilibrium wasting copper, electricity and occasionally burning homes and offices due to requiring twice the current for the same power compared to a 220-250V AC system. And do not forget the need for surge protectors or the destruction from not using them. I would not be surprised if this particular bill lying in the street (but glued there by a very stable Nash equilibrium with multiple moving parts) would be worth trillions of dollars, if all effects are added up. In Europe, people managed to pry it off the street, but only after huge parts of the electric infrastructure were bombed to dust or otherwise destroyed in WW2. At that point, it became possible to upgrade, making use of the opportunity afforded by better polymer insulators for wires.
I agree with Taleb that the ability to benefit from destructive shocks is key to long-term survival. But in order to achieve it in practice, the system needs to be stuck in a suboptimal state, with trapped free energy that can only be released upon a destructive shock and not accessible otherwise. The system does rely on these shocks coming from time to time, but they do. Sometimes it is the shockwaves from something snapping inside the system. For example, if the system wastes some non-renewable resource in a crazy Nash equilibrium, when that resource runs out, it may well provide the needed shock that would jolt it to a more efficient Nash equilibrium.
I’ve written a primer on voting theory that’s relevant to the voting method issues discussed here.
I’m not sure if I’m more Simplicio or more the Visitor, but … the political side of it doesn’t seem that hard to fix. At least, I’ve been advocating a set of improvements to futarchy that address all the political-structure inadequacies discussed here, as well as several others not discussed here. I know it’s too much to hope that it’s free of such inadequacies … though I still hope, because I explicitly designed it with the intent to be free of them. So I decided to write up a description of it suitable for this crowd: Ophelia, a weapon against Moloch.
Since political inadequacy seems to underlie many other types of inadequacy, maybe I should reconsider making an alpha version Ophelia app. In the meantime, if any of you wish to criticize flaws in the idea, or point me to a better idea, please do.
When creating a new governance system, don’t focus on creating something that’s supposed to work at the scale of a country.
Focus on something that works on a smaller scale. Student self-goverance at universities should be the perfect test bed for new governance models.
Er, is that agreement or an objection? It reads like an objection to me, though that could be the lack of body language. But the content agrees with the post, which explicitly states at both the beginning and the end that the system is designed to start very small and then grow.
This means that the organization with the best machine learning algorithm to estimate the bill score gets a lot of political power.
The same goes for the actual making of predicitions.
This looks like short-term effects of the bill become more important than it’s long-term effects.
There are likely some interesting Goodhard’s law problems where issues that benefit special interest groups aren’t asked about in the polling questions and thus there’s favors to be exchanged with those interests groups.
I would expect a few big organisations to arrise that have heavy machine learning capabilities and that hold the power about bill making.
In general the system is likely sufficiently intransparent that it’s hard to understand whats happening and how the big organisations use their power to get benefits but they likely will be able to get benefits.
It’s true I omitted the possibility of expending votes at a Vickrey auction level instead of an actual-bid level, so I grant the possibility that, if only one side had good polling data (implausible as that is), then they might buy votes a small fraction more cheaply. However, the “proportional influence” criterion is where most of the power of the system is. - i.e. How would one side actually increase their power long term, since the redistribution of spent votes eliminates their advantage after a single bill, which could then be cheaply repealed by the opposition? And since they’d still have to be maximizing the totals from the polling, would successfully gaining strategic advantage over bills have any downsides?
Mostly true, but missing two significant refinements: (1) What actually matters is the effects of the entire legal code, to which the most recent bill ought to have made at most a small adjustment. (2) It’s the task of the questions to ask about short-term judgments of long-term effects. No system can select for long-term effects that aren’t predictable until the long-term has already arrived.
Entirely fair, though I’d propose that transparency is not a critical difference between a system like this, that can only be fully understood by people who read up on it, versus a system like the U.S. election laws, that can’t be fully understood even by people who read up on it. The U.S. election laws do have the appearance of simplicity as long as one explains First Past The Post and nothing more.
Could someone help me understand this section?
Did Simplicio mean to say GREEN politicians? The gerrymandering arrangement sets the bar lower for Greens to win (55%) than Oranges (100%), so Green re-election is what’s nearly guaranteed.
No, Simplicio means Orange politicians. All districts are ‘safe,’ in that the majority party in the district is likely to win. But the orange districts are extremely safe—so much so that one expects the orange politicians to become out of touch with their voters. The theory is that competition is what forces politicians to actual pay attention to what voters care about, and to steal issues from other politicians that might be generally popular.
Like ChristianKl points out, this mostly ignores primaries, but note that even in primaries the Orange politicians only really need to care about internecine Orange issues, rather than developing a nationally relevant platform. If they have the power to knock potential contenders out of the Orange party, this means they they can route around the voters entirely. (Attempts by the Greens to do this in 55% Green districts would lead to a Green loss.)
Individual politicians care about getting elected in addition to caring about their party having the majority. The gerrymandering means that in the district of an orange politician you would get 90% oranges if that would be possible to gerrymander.
In practice EY ignores that there are primaries. Orange politicians do have to face primary challengers and pander to their base. This means that there are forces to have more extreme orange politicians than you would otherwise get.
All these depictions about multi-tiered stages through which action & incentives have to pass—like venture capital going thru multiple rounds A → B → C, or the recursion of “it’s more than just what you think, but also what you think others think”, and “what will the journalists say about what the politicians say outside the overton window” etc. seem to me to have a similar theme underlying them all, that ‘the further distance you are away, the more screwed up and misaligned the effects are’, and the more we’re locked-in to weird states of inadequate equilibria. And this reminds me of an analogy in Physics. (And I HOPE this is just a casual analogy, and not indicative of some deeper ‘spooky-inadequacy-at-a-distance’ that suggests we’re screwed in this matter all the way down to physics itself!) Like a chair, when you drag it across the kitchen floor and the legs vibrate with jerkiness against the floor and make a screech sound. Or how we have earthquakes—the plates are moving continuously, but the earthquakes occur in jerks, building up pressure to finally jump over a threshold. Or like thermal heating when your heater is heating up and it ticks loudly as the metal expands in jerks. All of this is underlied by the one physics phenomenon that static friction happens to be greater than kinetic friction. So even when you move that chair perfectly continuously and gradually at the top, the further distance out along a chain of steps away (in this case atoms), the more things get jerky at the endpoint. Kind of like the gradual winds of change of public opinion only getting realized in jerks, and revolutions, and rapid phase-transitions over a threshold. Anyway, that’s the thought I couldn’t shake while reading this chapter. I cling to hope that some of these meta-level dynamics to our problems depicted by EY here are remediable somehow… and we’re not predeterminedly screwed on a physical level, unable to confront them. Clearly we need to kick the shit out of these inadequacies! Maybe a rough rule of thumb could be to get up close and personal, right up next to them, not out at a distance of 3 recursive middle-men away (IF even possible to change at all) to lower entry-barriers and have a better chance of the ‘just-snap-out-of-it’? Proposal: 1. Get a Friendly Aligned-AI in power, 2. Go thru the whole tree-of-fruits minimizing the inadequacy-distance away systematically, and 3. If you got past step 1 and are both still alive and also not a paperclip, you’re in the clear, by Anthropic Principle. (Also don’t worry about step 2). So, uh.. just step 1 then.
I’m not sure the recursive argument even fully works for the stock market, these days—I suspect it’s more like a sticky tradition that crudely mimic the incentive structure that used to exist, like a parasitic vine that still holds the shape of the rotted-away tree it killed. When there’s any noise, recursion amplifies it with each iteration: a 1-year lookahead to a 1-year lookahead might be almost the same as a 2-year-lookahead, but it’s slightly skewed by wanting to take into account short-scale price movements and different risk and time discounting. By the the time you get to a 1-year lookahead to a 1-year lookahead to a...*10, it’s almost completely (maybe completely) decoupled from the actual 10-year lookahead, with no way to make money off of that decoupling.
I’d agree with Simplicio that voter’s “stupidity” as in “ignorance and inability to discern correctly on even issues where a scientific consensus has been reached and it really feels like a good, intuitive idea to make an internet search of ten minutes and check what the most accredited institutions are saying on the matter” would interact a lot with the border of the Overton window.
If 90% of the voters were able to mock any “stupid” idea suggested, moving out of the Overton window by going down with the quality of idea discussed would be plain suicide, moving up would sometime reward. Attempts to shift the Overton window downward, such as “hey, let’s completely go against what (insert science field) says about (insert important issue), and lets (choose between prohibiting particular subgroup therapies even if science says it’s really a good idea to provide said therapies/talk against preventing key crisis that will product unbelievable damage in short term future/suggest completely unfounded model about social issue x works and propose solution unrelated to any actual finding on the matter that has a track history of failures)” would be harshly punished by the voters, while right now these seem to be roughly 30% of politics discussed.
Still, I guess Cecie’s theory can explain the source of this “stupidity” with systemic failures that happen in other parts of society such as information and education, while if we just ascribe this to widespread individual “stupidity” and “sheepness” we are not less confused, but perhaps more so.
Does anyone have a theory about why this is the case? Thinking out loud:
voting systems: I guess any mention from a politician would be immediately dismissed as agenda-based. And well, probably any mention from anyone. Making changes to political systems also has a Chesterton’s fence component: we know how our system behaves, we don’t know how this new system would behave, and we know from history that we should be quite happy to have political systems that kind of work and haven’t already led us to totalitarianism. I’m not sure this aspect contributes to making these ideas outside of the Overton window, though.
academic science being focused more on prestige than on producing knowledge: it’s about prestige and social status, which are a bit taboo? Fear of fueling science deniers? Respectable newspapers have themselves been pursuing prestige for decades?
other relevant meta-problems?
In general, if you talk about changing the rules of the game you’re playing in your personal career, you’ll be dismissed as making up new rules just to get an edge; if you talk about rules you’re not playing, you’re not an expert.