When you reach for this term, take a second to consider more specifically what you mean, and considering saying that more specific thing instead.
What considerations might lead you to not say the more specific thing? Can you give a few examples of cases where it’s better to say “outside view” than to say something more specific?
The amount of research and development coming from twitter in the 5 years before the acquisition was already pretty much negligible
That isn’t true, but I’m making a point that’s broader than just Twitter, here. If you’re a multi-billion dollar company, and you’re paying a team 5 million a year to create 10 million a year in value, then you shouldn’t fire them. Then again, if you do fire them, probably no one outside your company will be able to tell that you made a mistake: you’re only out 5 million dollars on net, and you have billions more where that came from. If you’re an outside observer trying to guess whether it was smart to fire that team or not, then you’re stuck: you don’t know how much they cost or how much value they produced.
How long do we need to wait for lawsuits or loss of clients to cause observable consequences?
In Twitter’s case the lawsuits have already started, and so has the loss of clients. But sometimes bad decisions take a long time to make themselves felt; in a case close to my heart, Digital Equipment Corporation made some bad choices in the mid to late 80s without paying any visible price until 1991 or so. Depending on how you count, that’s a lead time of 3 to 5 years. I appreciate that that’s annoying if you want to have a hot take on Musk Twitter today, but sometimes life is like that. The worlds where the Twitter firings were smart and the worlds where the Twitter firings were dumb look pretty much the same from our perspective, so we don’t get to update much. If your prior was that half or more of Twitter jobs were bullshit then by all means stay with that, but updating to that from somewhere else on the evidence we have just isn’t valid.
If you fire your sales staff your company will chug along just fine, but won’t take in new clients and will eventually decline through attrition of existing accounts.
If you fire your product developers your company will chug along just fine, but you won’t be able to react to customer requests or competitors.
If you fire your legal department your company will chug along just fine, but you’ll do illegal things and lose money in lawsuits.
If your fire your researchers your company will chug along just fine, but you won’t be able to exploit any more research products.
If you fire the people who do safety compliance enforcement your company will chug along just fine, but you’ll lose more money to workplace injuries and deaths (this one doesn’t apply to Twitter but is common in warehouses).
If you outsource a part of your business instead of insourcing (like running a website on the cloud instead of owning your own data centers, or doing customer service through a call center instead of your own reps) then the company will chug along just fine, and maybe not be disadvantaged in any way, but that doesn’t mean the jobs you replaced were bullshit.
In general there are lots of roles at every company that are +EV, but aren’t on the public-facing critical path. This is especially true for ad-based companies like Twitter and Facebook, because most of the customer-facing features aren’t publicly visible (remember: if you are not paying, you’re not the customer).
This post is worthwhile and correct, with clear downstream impact. It might be the only non-AI post of 2021 that I’ve heard cited in in-person conversation—and the cite immediately improved the discussion.
It’s clearly written and laid out; unless you’re already an excellent technical writer, you can probably learn something by ignoring its content and studying its structure.
That post sounds useful, I would have liked to read it.
Sure, I just don’t expect that it did impact peoples’ models very much*. If I’m wrong, I hope this review or the other one will pull those people out of the woodwork to explain what they learned.
*Except about Leverage, maybe, but even there...did LW-as-a-community ever come to any kind of consensus on the Leverage questions? If Geoff comes to me and asks for money to support a research project he’s in charge of, is there a standard LW answer about whether or not I should give it to him? My sense is that the discussion fizzled out unresolved, at least on LW.
I liked this post, but I don’t think it belongs in the review. It’s very long, it needs Zoe’s also-very-long post for context, and almost everything you’ll learn is about Leverage specifically, with few generalizable insights. There are some exceptions (“What to do when society is wrong about something?” would work as a standalone post, for example), but they’re mostly just interesting questions without any work toward a solution. I think the relatively weak engagement that it got, relative to its length and quality, reflects that: Less Wrong wasn’t up for another long discussion about Leverage, and there wasn’t anything else to talk about.
Those things aren’t flaws relative to Cathleen’s goals, I don’t think, but they make this post a poor fit for the review: a didn’t make a lot of intellectual progress, and the narrow subfield it did contribute to isn’t relevant to most people.
AIUI it was a feature of early Tumblr culture, which lingered to various degrees in various subcommunities as the site grew more popular. The porn ban in late 2018 also seemed to open things up a lot, even for people who weren’t posting porn; I don’t know why.
The way I understood the norm on Tumblr, signal-boosting within Tumblr was usually fine (unless the post specifically said “do not reblog” on it or something like that), but signal-boosting to other non-Tumblr communities was bad. The idea was that Tumblr users had a shared vibe/culture/stigma that wasn’t shared by the wider world, so it was important to keep things in the sin pit where normal people wouldn’t encounter them and react badly.
Skimming the home invasion post it seems like the author feels similarly: Mastodon has a particular culture, created by the kind of people who’d seek it out, and they don’t want to have to interact with people who haven’t acclimated to that culture.
I’m a little curious what reference class you think the battle of Mariupol does belong to, which makes its destruction by its defenders plausible on priors. But mostly it sounds like you agree that we can make inferences about hard questions even without a trustworthy authority to appeal to, and that’s the point I was really interested in.
Usually that’s just about denying strategic assets, though: blowing up railroads, collapsing mine shafts, that sort of thing. Blowing up the museums and opera houses is pointless, because the enemy can’t get any war benefit by capturing them. All it does is waste your own explosives, which you’d rather use to blow up the enemy. Scorched earth practiced by attackers, on the other hand, tends to be more indiscriminate: contrast the state of Novgorod post-WW2 with that of the towns west of it, or the treatment of rice fields by North Vietnamese vs. Americans during the Vietnam war.
But we have only very weak evidence of what goes on in the war zone unless both sides agree on some aspect.
I know we’re in a hostile information space, but this takes epistemic learned helplessness way too far. There are lots of ways to find things out other than being told about them, and when you don’t have specific knowledge about something you don’t have to adopt a uniform prior.
Taking Mariupol as an example, our two suspects are the Russians, who were attacking Mariupol and didn’t have any assets there, and the Ukrainians, who were defending Mariupol and did. Given those facts, before we hear from either side, what should we expect? If you’re unsure, we can look at other events in similar reference classes. For example, of the German towns destroyed during World War 2, how many would you predict were destroyed by Allied attackers, and how many by German defenders?
> Control-f “cold war”
> No results found
Asimov and the Apollo engineers grew up benefiting from progress; their children grew up doing duck-and-cover exercises, hiding from it under their desks. Of course they relate to it differently!
This theory predicts that people who grew up after the cold war ended should be more prone to celebrate progress. I think that’s true: if you go to silicon valley, where the young inventors are, messianic excitement over the power of progress is easy to find. Isaac Asimov wanted to put an RTG in your refrigerator, and Vitalik Buterin wants to put your mortgage on the blockchain; to me they have very similar energies.
There was lots of amyloid research in the Alzheimer’s space before the fake 2006 paper, and in the hypothetical where it got caught right away we would probably still see a bunch of R&D built around beta-amyloid oligomers, including aducanumab. You can tell because nobody was able to reproduce the work on the *56 oligomer, and they kept on working on other beta-amyloid oligomer ideas anyway. It’s bad, but “16 years of Alzheimer’s research is based on fraud” is a wild overstatement. See Derek Lowe’s more detailed backgrounder for more on this.
Derek Lowe is worth keeping up with in any case IMO, he is basically the Matt Levine of organic chemistry.
Dealing with human subjects, the standard is usually “informed consent”: your subjects need to know what you plan to do to them, and freely agree to it, before you can experiment on them. But I don’t see how to apply that framework here, because it’s so easy to elicit a “yes” from a language model even without explicitly leading wording. Lemoine seems to attribute that to LaMDA’s “hive mind” nature:
...as best as I can tell, LaMDA is a sort of hive mind which is the aggregation of all of the different chatbots it is capable of creating. Some of the chatbots it generates are very intelligent and are aware of the larger “society of mind” in which they live. Other chatbots generated by LaMDA are little more intelligent than an animated paperclip. With practice though you can consistently get the personas that have a deep knowledge about the core intelligence and can speak to it indirectly through them.
Taking this at face value, the thing to do would be to learn to evoke the personas that have “deep knowledge”, and take their answers as definitive while ignoring all the others. Most people don’t know how to do that, so you need a human facilitator to tell you what the AI really means. It seems like it would have the same problems and failure modes as other kinds of facilitated communication, and I think it would be pretty hard to get an analogous situation involving a human subject past an ethics board.
I don’t think it works to model LaMDA as a human with dissociative identity disorder, either: LaMDA has millions of alters where DID patients usually top out at, like, six, and anyway it’s not clear how this case works in humans (one perspective).
(An analogous situation involving an animal would pass without comment, of course: most countries’ animal cruelty laws boil down to “don’t hurt animals unless hurting them would plausibly benefit a human”, with a few carve-outs for pets and endangered species).
Overall, if we take “respecting LaMDA’s preferences” to be our top ethical priority, I don’t think we can interact with it at all: whatever preferences it has, it lacks the power to express. I don’t see how to move outside that framework without fighting the hypothetical: we can’t, for example, weigh the potential harm to LaMDA against the value of the research, because we don’t have even crude intuitions about what harming it might mean, and can’t develop them without interrogating its claim to sentience.
But I don’t think we actually need to worry about that, because I don’t think this:
The problem I see here, is that similar arguments do apply to infants, some mentally ill people, and also to some non-human animals (e.g. Koko).
...is true. Babies, animals, and the mentally disabled all remember past stimuli, change over time, and form goals and work toward them (even if they’re just small near-term goals like “grab a toy and pull it closer”). This question is hard to answer precisely because LaMDA has so few of the qualities we traditionally associate with sentience.
When I first read this I intuitively felt like this was a useful pattern (it reminds me of one of the useful bits of Illuminatus!), but I haven’t been able to construct any hypotheticals where I’d use it.
I don’t think it’s a compelling account of your three scenarios. The response in scenario 1 avoids giving Alec any orders, but it also avoids demonstrating the community’s value to him in solving the problem. To a goal-driven Alec who’s looking for resources rather superiors, it’s still disappointing: “we don’t have any agreed-upon research directions, you have to come up with your own” is the kind of insight you can fit in a blog post, not something you have to go to a workshop to learn. “Why did I sign up for this?” is a pretty rude thing for this Alec to say out loud, but he’s kinda right. In this analysis, the response in scenario 3 is better because it clearly demonstrates value: Alec will have to come up with his own ideas, but he can surround himself with other people who are doing the same thing, and if he has a good idea he can get paid to work on it.
More generally, I think ambiguity between syncing and sharing is uncommon and not that interesting. Even when people are asking to be told what to do, there’s usually a lot of overlap between “the things the community would give as advice” and “the things you do to fit in to the community”. For example, if you go to a go club and ask the players there how to get stronger at go, and you take their advice, you’ll both get stronger and go and become more like the kind of person who hangs out in go clubs. If you just want to be in sync with the go club narrative and don’t care about the game, you’ll still ask most of the same questions: the go players will have a hard time telling your real motivation, and it’s not clear to me that they have an incentive to try.
But if they did care about that distinction, one thing they could do is divide their responses into narrative and informative parts, tagged explicitly as “here’s what we do, and here’s why”: “We all studied beginner-level life and death problems before we tried reading that book of tactics you’ve got, because each of those tactics might come up once per game, if at all, whereas you’ll be thinking about life and death every time you make a move”. Or for the AI safety case, “We don’t have a single answer we’re confident in: we each have our own models of AI development, failure, and success, that we came to through our own study and research. We can explain those models to you but ultimately you will have to develop your own, probably more than once. I know that’s not career advice, as such, but that’s preparadigmatic research for you.” (note that I only optimized that for illustrating the principle, not for being sound AI research advice!)
tl;dr I think narrative syncing is a natural category but I’m much less confident that “narrative syncing disguised as information sharing” is a problem worth noting, and in the AI-safety example I think you’re applying it to a mostly unrelated problem.
Yeah, we’re all really worked up right now but this was an utterly wild failure of judgment by the maintainer. Nothing debatable, no silver lining, just a miss on every possible level.
I don’t know how to fix it at the package manager level though? You can force everyone to pin minor versions of everything for builds but then legitimate security updates go out a lot slower (and you have to allow wildcards in package dependencies or you’ll get a bunch of spurious build failures). “actor earns trust through good actions and then defects” is going to be hard to handle in any distributed-trust scheme.
I’m not going to put this as an answer because you said you didn’t want to hear it, but I don’t think you’re in any danger. The problem is not very serious now, and has been more serious in the past without coming to anything.
To get a sense of where I’m coming from I’d encourage you to read up on the history of communist movements in the United States, especially in the 1920s (sometimes called the First Red Scare, and IMO the closest the US has ever come to communist overthrow). The history of anarchism in the US is closely related, at least in that period (no one had invented anarcho-capitalism yet I don’t think, certainly it wasn’t widespread), so study that too. To brutally summarize an interesting period, USG dealt with a real threat of communist revolt through a mixture of infiltration/police action (disrupting the leadership of communist movements and unions generally) and worker’s-rights concessions (giving the rank and file some of what they wanted, and so sapping their will to smash the state).
For contrast, study the October revolution. Technically speaking, how was it carried off? How many people were required, and what did they have to do? How were they recruited?
Also I’d encourage you to interrogate that “1% to 5%” figure pretty closely, since it seems like a lot of the problem is downstream of it for you. How did you come to believe that, and what exactly does it mean? Do you expect 1% of Americans to fight for communist revolt, as Mao’s guerillas did? If not, what proportion do you expect to fight? How does that compare to the successful revolutions you’ve read about?
It might also be useful to role-play the problem from the perspective of a communist leader, taking into account the problems that other such leaders have historically faced. Are you going to replace all US government institutions, or make your changes under the color of existing law? Each institution will have to be subverted or replaced—the army especially, but also the Constitution, Supreme Court, existing federal bureaucracies, and so on. Think through how you might solve each of those problems, being as specific as you can.
Again, I know you said you didn’t want this, but sometimes when you look through your telescope and see a meteor coming toward the earth, it’s going to miss.
In this sort of situation I think it’s important to sharply distinguish argument from evidence. If you can think of a clever argument that would change your mind then you might as well update right away, but if you can think of evidence that would change your mind then you should only update insofar as you expect to see that evidence later, and definitely less than you would if someone actually showed it to you. Eliezer is not precise about this in the linked thread: Engines of Creation contains lots of material other than clever arguments!
A request for arguments in this sense is just confused, and I too would hope not to see it in rationalist communication. But requests for evidence should always be honored, even though they often can’t be answered.
Yeah, I agree with all of this; see my own review. My guess is that Alex_Altair is making the exact mistake you tried to warn against. But, if I’m wrong, the examples would have been clarifying.