Before you say “left vs right”, consider that e.g. veganism is coded left-wing, but we don’t hear about companies turning vegan under external pressure
I think this is definitely a thing that happens, and actually is one of the primary strategies of animal activists these days. (Granted, the current stage of that strategy is more like “going cage free” or “meatless mondays”).
I definitely did not intend to make either an airtight or exhaustive case here. I think coherence and consistency are good for a number of reasons, and I included the ones I was most confident in, and felt like I could explain quickly and easily. (The section was more illustrative than comprehensive)
This response will not lay out the comprehensive case, but will try to answer my current thoughts on some specific questions. (I feel a desire to stress that I still don’t consider myself an expert or even especially competent amature on this topic)
Second, the only case for Coherence is that reasons that coherence helps you make trade with your future self
That’s actually not what I was going for – coherence can be relevant in the moment (if I had to pick, my first guess is that coherence is more costly in the moment and inconsistency is more costly over time, although I’m not sure I was drawing a strong distinction between them)
If you have multiple goals that are at odds, this can be bad in the immediate moment, because instead of getting to focus on one thing, you have to divide up your attention (unnecessarily) between multiple things that are at odds. This can be stressful, it can involve cognitive dissonance which makes it harder to think, and it involves wasted effort
Have you looked into the workings of large organisations outside of North America?
Do you actually mean “North America” or “United States in particular”?
Recent developments indicate an increasing simulacrum level
I noticed wanting this to be a link, and the I noticed, “I think someone just really needs to write an updated Simulacrum post that’s a lot more clear.”
Do you have a suggestion for keeping the wrong behaviors out, without keeping the wrong people out? (In general I agree with ‘give people feedback’, but ‘deception’ is one of the specific cases where I’m much less optimistic about that. They were willing to deceive in the first place, how do you trust that they didn’t just get better at deceiving rather than reform?)
I’m not sure which of these posts is a subset of the other:
The Backbone Bottleneck
The Leadership Bottleneck
> If you’re a king with 5 districts, and you have 20 competent managers who trust each other… one thing you can do is assign 4 competent managers to each fortress, to ensure the fortress has redundancy and resilience and to handle all of its business without any backstabbing or relying on inflexible bureaucracies. But another thing you can do is send 10 (or 15!) of the managers to conquer and reign over *another* 5 (or 15!) districts.
> This is bad if you’re one of the millions of people who live in the kingdom, who have to contend with werewolves.
> It’s an acceptable price to pay if you’re actually the king. Because if you didn’t pay the price, you’d be outcompeted by an empire who did. And meanwhile it doesn’t actually really affect your plans that much.
...The key instinct is that any price that can be paid to be stronger or more competitive, must be paid, therefore despair: If you didn’t pay the price, you’d be out-competed by someone who did. People who despair this way often intuitively are modeling things as effectively perfect competition at least over time, which causes them to think that everything must by default become terrible, likely right away.
[...]
Kingdoms don’t reliably expand to their breaking points.
Anthropics vs Goals
I didn’t get around to replying to this until today, but this wasn’t my main point and I think it’s pretty important.
The issue isn’t whether you’ll fail to achieve your goals if you don’t expand. The issue is “from an anthropic reasoning perspective, what sort of world will most people live in?”
I have shifted some of my thinking around “you’ll be outcompeted and therefore it’s in your interest to expand”. I think I agree with “it’s generally not worth trying to be the winner-take-all winner, because a) you need to sacrifice all the things you cared about anyway, b) even if you do, you’re not actually likely to win anyway.”
But that was only half the question – if you’re looking around the world, trying to build a model of what’s going on, I think the causal explanation is that “organizations that expand end up making up most of the world, so they’ll account for most of your observations.”
The reason this seems important is, like, I see you and Benquo looking in horror at the world. And… it is a useful takeaway that “hmm, I guess I don’t need to expand in order to compete with the biggest empires in order to be happy/successful/productive, I can just focus on making a good business that delivers value and doesn’t compromise it’s integrity.” (And having that crystallized has been helpful to my own developing worldview)
Nonetheless… the world will continue to be a place you recoil in horror from until somehow, someone creates something either stops mazes, or outcompetes them, or something.
Breaking Points vs Realistic Tradeoffs
I also disagree with the characterization “kingdoms don’t expect to breaking point.”
The original concept here was “why do people have a hard time detecting obfuscators and sociopaths?”. A realistic example (to be clear I don’t know much about medieval kingdoms), is a corporation that ends up creating multiple departments (i.e. hiring a legal team), or expanding to new locations.
This doesn’t mean you expand to your breaking point – any longterm organization has to contend with shocks and robustness. The organizations I expect to be most successful will expand carefully, not overextending. But if you’re asking the question “why are there obfuscators everywhere?”, I think the answer is because the relative profitability of extinguishing obfusctators, vs. not worrying as much about it, points towards the latter direction.
This is, in part, because extinguishing obfuscating or other mazelike patterns is a rare, high skill job that, like, even small organizations don’t usually have the capacity to deal with. I think if you can make it much cheaper, it’s probably possible to shift the global pattern. But I think the status quo is that the profit-maximizing thing to do focus on expansion over being maze-proof, and there’s a lot of profit-maximizing-entities out there.
It’s not worth it for the king to try to expand to take over the world. It still seems, for many kings in many places, that expanding reasonably, robustly, is still the right strategy given their goals (or at least, they think it’s their goal, and you’d have your work cut out for you convincing them otherwise), and that meanwhile worrying about werewolves in the lawyer department is probably more like a form of altruism than a form of self-interest.
Or, reframing this as a question (since I’m honestly not that confident)
If your inner circle is safe, how much selfish reason does a CEO have to make sure the rest of the organization is obfuscator-proof?
This still sounds like it’s the bucket of “I have turned on ‘cooking’ as a broader action that I’m doing”. (which is not always a dealbreaker, I enjoy cooking sometimes, but often a dealbreaker no matter the action)
Having to clean an eggbeater is a dealbreaker for me.
(that said I don’t really use fancy butter either)
Welcome!
This is probably the post I got the most value out of in 2018. This is not so much because the precise ideas (although I have got value out of the principle of meta-honesty, directly), but because it was an attempt to understand and resolve a confusing, difficult domain. Eliezer explores various issues facing meta-honesty – the privilege inherent in being fast-talking enough to remain honesty in tricky domains, and the various subtleties of meta-honesty that might make it too subtly a set of rules to coordinate around.
This illustration of “how to contend with potential dealbreakers for a new philosophical tool” was more useful to me than the tool itself. I learned a bit about how to become more honest from this post, and more generally, how to improve my policies and coordination procedures in a careful, sane way.
I don’t that currently, meta-honesty is a good concept to coordinate around in large groups. It may or may not be too complicated to ever work on the 100-300 person scale. If it could work there, I think there is more distillation work that’s needed.
I do think it works fairly well among individuals who know each other well and have the time to build up high resolution models of each other.
I can’t remember whether I’ve explicitly used meta-honesty while communicating with others, but I’ve built up some internal models that rely more broadly on “meta-trust”. (I have some posts-in-the-works ironing out what that means)
As is Moby Dick though it is not explicit in the latter whereas the metaphor is explicitly declared in the former
I’d be interested in hearing more thoughts about this.
I think I was more resigned to it.
I just re-read this sequence. Babble has definitely made its way into my core vocabulary. I think of “improving both the Babble and Prune of LessWrong” as being central to my current goals, and I think this post was counterfactually relevant for that. Originally I had planned to vote weakly in favor of this post, but am currently positioning it more at the upper-mid-range of my votes.
I think it’s somewhat unfortunate that the Review focused only on posts, as opposed to sequences as a whole. I just re-read this sequence, and I think the posts More Babble, Prune, and Circumambulation have more substance/insight/gears/hooks than this one. (I didn’t get as much out of Write). But, this one was sort of “the schelling post to nominate” if you were going to nominate one of them.
The piece as a whole succeeds very much as both Art as well as pedagogy.
I think it’s technically right, but something like “company politics is more horrifying than you think in subtle ways” that people will tend to gloss over, or something.
Nod. Mingyuan probably has more specific answers here, but I do basically endorse her overall approach and have some thoughts about it.
I think it’s possible (and desirable) to have a single coherent vision for the overall event, while still having personal stories (and indeed that’s what ended up happening here). I think the process was some combo of:
figure out overall shape of the arc
in some cases write specific things for pieces of it
in some cases reach out to people to write things that fit into that part of the arc
in some cases write things, and then send them to people and say ‘hey, can you take this basic idea but write something of your own that’s in your own voice’? (such as Tessa’s speech)
in some cases, find people who had already written things that fit well (such as habryka’s eulogy)
#4 was an option I hadn’t really considered before and I think worked well.
The problem with people writing personal stories independently is they often end up covering fairly similar ground, in a way that ends up a combo of repetitive and disjointed. (Instead of a smooth arc that flows down and then up, you get some weird repetitive motions in the middle, and meanwhile the whole thing ends up with a bit of a ‘designed by committee’ feel. That doesn’t always happen but I’ve seen it happen a few times)
Part of the issue is that there’s a cluster of issues that overlap and are kinda fuzzy and if you try to pin them down you might miss the forest for the trees. I think (unfortunately) that reading the full set of Quotes on Moral Mazes.
I think it’s possible to do a better job summarizing than Zvi has currently done. My current attempt at the cluster of patterns are:
Middle managers end up focusing on success within the ecosystem of middle-management, which is increasingly (or entirely) divorced from any object level value that the organization is actually producing.
Because it’s hard to evaluate middle managers, the evaluation ends up being almost entirely ‘politics’.
This self-reinforces for the reasons Zvi describes in this post.
The ecosystem ends up punishing attempts to be communicate clearly about ethics, and about the object level output of the company (since often those are politically inconvenient)
The ecosystem ends up forcing you to self-modify to select all your hobbies/life/politics/family around company politics, and in the process also self-modify to be more ethically comfortable with how the maze is set up.
woo!
This might be a little off-topic, but are there any ideas / resources about how to organize a rationalist winter solstice if (i) you don’t have a lot of time or resources (ii) you expect at most a small core of people which are well-familiar with the rationalist memeplex and buy into the rationalist ethos, plus some number of people who only partway there or are merely curious. Or, is it not worth trying under these conditions?
I think it’s actually quite sad/bad that it comes across as “it’s not worth doing a Solstice if you don’t have lots of resources and expect a small turnout.” In fact, this was the original use-case for which Solstice was designed.
(I have a rant about this over here, which I intend to turn into a more polished essay sometime. tl;dr: Big Community Solstices are quite valuable, but they are not the only way to run a Solstice. Over the longterm [i.e. decades] I would measure most of the health of Solstice-as-an-insitution in terms of how many small solstices there are. Small solstices should have pretty different expectations of polish. They also offer the opportunity to have more interpersonal connection)
How to handle ‘some people have bought into the rationalist ethos and some have not’ is a question that depends a lot on the percentages, but I think there are approaches that work pretty fine in that context.
This is the small Solstice I run when I want to just roll out of bed and have a small Solstice without much prep. It does depend a bit on my ability to confidently lead all the songs, which would require some up front work to learn. It relies pretty minimally on the rationalist memeplex. I think there are some (relatively minor) changes you could make that would make it easier on a completely new organizer. (I’m happy to skype or chat with new organizers to help them get oriented, PM me if that’s helpful)
(that version is designed for being outdoors, but mostly works fine if you hold it in a living room and change a couple words)
I do have on my longterm todo list to put together a collection of “off-the-shelf Solstices” that serve a few different use-cases, and better instruction on how to customize them. Meanwhile, here’s the things I’ve written so far.
This sounds like the right mechanism to me.
I do think “why do some things become politically fashionable?” is an important question. I think the answer to that is basically a whole other subfield just as complicated as the “how do mazes form?” question. But in answer to:
I think the answer is just “all of them” and “it depends.”