I’ve been a LessWrong organizer since 2011, with roughly equal focus on the cultural, practical and intellectual aspects of the community. My first project was creating the Secular Solstice and helping groups across the world run their own version of it. More recently I’ve been interested in improving my own epistemic standards and helping others to do so as well.
Raemon(Raymond Arnold)
Karma: 31,123
My own experience is somewhat like Linch’s here, where mostly I’m vaguely aware of some things that aren’t my story to tell.
For most of the past 9ish years I’d found Leverage “weird/sometimes-offputting, but not obviously moreso than other rationality orgs.” I have gotten personal value out of the Leverage suite of memes and techniques (Belief Reporting was a particularly valuable thing to have in my toolkit).
I’ve received one bit of secondhand info about “An ex-leverage employee (not Zoe) had an experience that seemed reasonable to describe as ‘the bad kind of cult that was actually harmful’.” I was told this as part of a decisionmaking process where it seemed relevant, and asked not to share it further in the past couple years. I think it makes sense to share this much meta-data in this context.
Periodically this book gets mentioned to me, and I try reading it and bounce off. I’d be interested in someone going in more detail into the parts that seem sensible/interesting/tractable for mid-level organization projects.
(concretely: the Lightcone team is working longterm on building a campus, and I expect us to be more likely to make use of a good summary of this, or a strong recommendation that the whole thing is worth reading)
Mostly time and attention. This has been on the list of things the LessWrong team has considered working on and there’s just a lot of competing priorities.
This post (unfortunately fairly badly formatted on his new website) probably won’t be new to you but is another that I refer back to from time to time.
(rereading it, upon reflection I’m not actually sure if I agree with the primary thesis of the post exactly, but the framing resonates)
https://www.theferrett.com/2014/09/04/the-answer-that-destroys-all-our-futures/
Man, this fills me with some creeping dread at how many complex problems need to be solved in order for the future to not be dystopic.
Also, wouldn’t being forced to retreat entirely to your “home” qualify as horrible conditions? That’s solitary confinement, no?
Depending on setup you can probably invite other people into your home.
Two anecdotes:
...
I recall when I was 18 years old or so, and I’d been arguing with a very religious friend throughout high school. I would rehearse arguments with him in my head, preparing for the next time we’d meet and I’d tell him all the reasons I thought his beliefs didn’t make sense.
And for the first couple years of this, in the arguments in my head, I’d always say things like “have you considered point X” and imaginary-friend would say “oh, man, you’re right. I am wrong.” But then, eventually, I hit points where I’d say “what about X?” and then my imaginary friend would say “so? X doesn’t matter, because [counterargument]”.
This was a neat thing to discover about my ability to model people. (It also was relevant to the entire “does God exist?” debate – an eventually cruxy point for me is that you totally can build up simulations of people in your head, and I’d expect that to be hard to distinguish from God speaking to you)
...
More recently, I received benefit from asking my own future self for advice. (In fact, I asked multiple future selves who might evolve in different directions). One future self responded with some concrete, compassionate advice about how one of my coping mechanisms wasn’t actually helping with my core goals.
This seems probably right.
I’m interested in the obvious followup questions of “how hard does this check out over more than 2 examples? How strong is the effect size of surplus food / how much food do you need? Do you need enlightenment memes separately from the food? Is food abundance sufficient or do you also need other kinds of abundance?”
Okay. So to try to tie this better back to the OP.
I want to reconcile the fact that my “common sense” morality includes “don’t steal from people, even the outgroup”, but “seems like empires are basically built out of theft (among other things) and like they were a key piece of how I even got my morality in the first place.”
And I think I was okay with “in practice, the way I got my morality was messy and build on atrocities, but at least I have a sense of how to do things going forward.” But the notion that “in principle, it may not have even been possible to get my morality without it being built on atrocities” is somehow deeply unsettling to me.
And maybe what I’m supposed to do is just grieve for that and move on with my day.
But I think not. Because I think we’re not necessarily done with empires needing to be built on (what I consider) immoral actions, for the greater good. (Or: it will probably turn out that at least some immoral empires will generate more goodness than could have been generated alternatively)
But it’s not like commonsense morality is meaningless. Theft is still bad for all the reasons it’s normally bad. I don’t want to stop punishing people for theft. Empires (which includes various large corporations) aren’t automatically for the greater good, so the rule of “you get punished for theft unless you’re too powerful” doesn’t actually solve anything even if my.
So the point of the previous comment was to grope towards “what norm do I actually want everyone to try to enforce, that has a shot at successfully applying to empires? Ideally that empire leaders should actually want to help enforce to create a higher-order incentive landscape?”
In a simpler ecosystem of individual agents, I think a reasonable principle is something like “it’s potentially okay for Alice to violate some norms to gain resources and accomplish a goal, if Alice later ‘makes good’ by paying some appropriate fine (ideally to the people who were harmed by her norm violation).”
But, this is much harder to apply to...
...vague-blobs of agents (who is useful to blame for crimes against the Irish? The current prime minister? The descendants of Oliver Cromwell? Whichever lower level official carried out various actions?)
...agents who are now dead (is there any way I can meaningfully punish Oliver Cromwell?)
...messy situations with lots of back and forth small crimes that escalated (My vague history sense is that England tried to stamp out Catholicism in Ireland, but I’m assuming the original Catholic settlers tried to stamp out other native practices, and everyone probably did random cattle raiding or conquest in various directions)
...
Thinking through this led me to some potential solutions:
It feels more robust to reward people who do good* than to punish people who do bad. It’s neither fair nor particularly possible to punish people who hadn’t been part of my moral framework, who’s deeds are mostly lost to the mists of time (and this is symmetrical for future people who might not share my framework but judge me by theirs)
Hmm. I guess I don’t necessarily want to punish people who did wrong, but I would ideally like to compensate people who were wronged.
A thing I can aim for for myself is to be the sort of person who tracks when I end up violating what I think should be a norm, and work to make it right in some way.
A thing I can also aim for myself is to work to ensure that organizations I participate in are robust, coherent organizations that are easier to audit and assign credit in, so I don’t contribute to a vague blob that’s hard to hold accountable.
There is something nice about the virtue of “ensuring that there is a record”, and being legible to historians. This enables future people, who might have a lot more resources, to allocate some kind of future rewards for people who put effort into making their organizations legible/coherent/agenty, and who attempted to do good with their imperial winnings.
(sufficiently advanced posthumans can solve this by literally implementing simulated ancestor heaven, and if that turns out to be intractable, you can instead do things like “figure out what someone
While thinking through all of this, one thing I found myself thinking of were things like slave reparations. Up until 10 minutes ago I was thinking something like “I think this has made me more pro-reparations”. The thought I had just now was “actually, the problem with reparations right now is that America/UK/whoever are too much of a vague messy blob to institute any morally complex norm reconciliation, and the primary thing to focus on if you’re worried about righting-past-wrongs is to help build a more coherent, agenty world that’s capable of doing anything on purpose at all.”
Nod.
The questions you list seem sort of generated by “what causes good things?”. I’m not sure if this is what you mean but it suggests an engineering mindset to me. (i.e. lets understand how culture and civilization works so we can make it be better (by our current lights? or current meta-lights?)
I’m currently looking at this from the generator that output the Coordination Frontier sequence, and I think I’m looking for something like “what are good principles for handling defection in weird confusing circumstances”, in ways that I’d expect aliens to independently converge on.
(I think the set of things we cluster around the word ‘morality’ includes both “coordination” as well as “doing whatever it is you value that’s worth coordinating for”. I am talking here primarily about “coordination”, but the two things aren’t crisp enough in my head to fully distinguish them)
Rambly thoughts that’ll maybe become a blogpost later:
Say you have two agents that meet in the wilderness. They have different values. Maybe one wants to make Art and Babies, and one wants to make PaperClips. Or maybe they both want Art and Babies but they want their particular Art and Babies.
They want the same resources.
In this example let’s say they speak the same language.
I think if one agent is vastly more powerful than the other, and neither agent is some kind of evolved values for empathy/universal-altruism, then I think it’s overdetermined that Alice just stomps all over Bob and takes his stuff.
If Alice and Bob are comparably powerful, such that it’s not immediately obvious who would win, then they do some complicated negotiation. The negotiation scheme depends on local features of their respective cultures, and the environment. (The ‘negotiation’ might take the form of ‘go to war’, but that’s costly and hopefully there are other less costly options).
But my guess is that there are at least some schelling norms that aliens would independently discover. (possible examples: honesty, not shooting messengers, keeping promises
). They might each wish to strategically defect from those norms, but agree that they’re overall better off if they can bootstrap an enforcement system that changes the payoff for defection.I think the longer an agent/culture has had time to think, the more complicated norms they will have time to discover.
This is all background for a half-formed thoughts that feel relevant to this post:
1) I suspect that “Just Punishment” is a schelling norm. i.e. it’s okay to punish defectors. This can take different forms like “it’s okay to kill in self defense”, “it’s okay to use violence to defend against violence”, etc
2) I suspect that “being in good or bad standing, and having to ‘pay for damages done’ to get back into good standing”, is a schelling norm.
3) I think “figure out how to bootstrap to actual trust/honesty, so you can save a ton of cognitive effort that you’d spend lying, or checking if other people are lying”, is a schelling norm. (Where the bootstrapping might include periodic surprise inspections that punish you massively if you turn out to be lying)
4) I think “figure out what deal you wish you had made, if you end up in a situation where you didn’t think to make a deal in advance, and then act as if you had made that deal”, is a good norm in many situations. I think it’s easier among people who trust each other. I think among people who don’t trust each other it can still work fairly well in some cases. (This is a bit handwavy at this point and probably needs a blogpost to flesh out)
i.e. if Alice and Bob get into a big protracted fight and end up in weird blood feud, they might say “okay, wait, wait, hold up. Man, this sucks. We didn’t have any clear coordination principles or communication frames when we got into this mess. What do we wish we had done instead of getting into a stupid blood feud?”. Where the answer isn’t “hold hands and sing kumbaya”, but might be “we wish we’d taken stock of our actual negotiating positions and resolved the dispute in some less costly way..” (this isn’t a great example, I’ll see if I can come up with a better one later)
5) points 1-4 apply differently to small groups who all know each other, vs large nations with many actors who vaguely share cultures. Nations are sort of like agents, and sort of not. I’m a bit confused how to think about that. (See that part of Hamilton where Jefferson says “we should honor the commitment we made to the French. What would Lafayette think of you turning your back on them?” and Hamilton replying with “the actual people we made the deal with are all dead”)
6) “what deal do you wish you had made?” is a bit confusing when you’ve gained new principles (coordination-wise, or otherwise) in the meanwhile
7) in many cases, the cognitive effort of figuring out what “should” have happened is too complicated and costly and you have to just sort of declare bankrupcty on the whole thing. But, if you do that a lot you give up on certain types of trust that things will ever be “made right”, which limits what sort of deals you can make in the future.
...
...tying this back to the OP is still a bit confusing. Stopping there for now.
Depends what you’re comparing Second Price Auction to, though, right?
I agree with the point that there is probably a longterm new equilibrium if you start doing it all the time, but then I thought for 10 minutes about it and I’m not sure what outcome you’d be worried about.
In the case of roommates negotiating the value of each room: this happens all the time for every shared apartment.
If landlords got to see the outcome of those negotiations, it might cause them to raise rents more, but landlords already get a general sense of when they are underselling their property and raise rents when they can.
If roommates continuously rented together and wanted to extract value from each other, without a second price auction, roommates are still often incentivized to exaggerate preferences. The SPA just streamlines the negotiation and helps ensure minimal envy between room allocations. (It might get less “magically good seeming” in equilibrium, but it still seems better than many alternatives)
Nod. Apologies for the somewhat curt/patronizing tone in the previous comment. I think your point here is a fine point but is not really what I wanted the discussion on this particular post to be about. (It feels like it’s arguing against a position I don’t expect anyone here to actually hold)
But, I do appreciate you following my posts generally and look forward to other discussion on this post and others.
Yes, I wasn’t at all saying bigger circles are generically better. (I realize this can be a bit tricky since this post involves some songs/poetry, and is in mid-confused-existential-crisis where a bunch of different pieces of my worldview get briefly namedropped without being spelled out. But I deliberately left this post non-frontpaged because I wanted the discussion of it to center on people who’ve read the sequences and have a bunch of similar philosophical background as me)
Note that this is being paid, like, way more than nonprofits normally pay.
Upvoted, but I mostly trust you to write the post if it seems like there’s an interesting meaty thing worth saying.
How does The Circle grow, exactly?
I’m assuming you need enough raw physical material to cover your mouth, which means either you either need one big one or multiple small ones.
(update: I had some data loss stuff during the pandemic because I started using some older/less advanced programs to do graphics and level-design for Gather Town. But I haven’t had the issue with most text editors or photoshop or whatever because of autosaving)
(This account is the same Kerry btw, my guess is Kerry happened to try logging in with google, which doesn’t actually connect to existing accounts)
FYI I just interpreted it to mean “former staff member” automatically. (This is biased by my belief that CFAR has very few current staff members so of course it was highly unlikely to be one, but I don’t think it was an unreasonably weird reading)