The comments here are a storage of not-posts and not-ideas that I would rather write down than not.
Yesterday I noticed that I had a pretty big disconnect from this: There’s a very real chance that we’ll all be around, business somewhat-as-usual in 30 years. I mean, in this world many things have a good chance of changing radically, but automation of optimisation will not cause any change on the level of the industrial revolution. DeepMind will just be a really cool tech company that builds great stuff. You should make plans for important research and coordination to happen in this world (and definitely not just decide to spend everything on a last-ditch effort to make everything go well in the next 10 years, only to burn up the commons and your credibility for the subsequent 20).
Only yesterday when reading Jessica’s post did I notice that I wasn’t thinking realistically/in-detail about it, and start doing that.
Responding to Scott’s response to Jessica.
The post makes the important argument that if we have a word whose boundary is around a pretty important set of phenomena that are useful to have a quick handle to refer to, then
It’s really unhelpful for people to start using the word to also refer to a phenomena with 10x or 100x more occurrences in the world because then I’m no longer able to point to the specific important parts of the phenomena that I was previously talking about
e.g. Currently the word ‘abuser’ describes a small number of people during some of their lives. Someone might want to say that technically it should refer to all people all of the time. The argument is understandable, but it wholly destroys the usefulness of the concept handle.
People often have political incentives to push the concept boundary to include a specific case in a way that, if it were principled, indeed makes most of the phenomena in the category no use to talk about. This allows for selective policing being the people with the political incentive.
It’s often fine for people to bend words a little bit (e.g. when people verb nouns), but when it’s in the class of terms we use for norm violation, it’s often correct to impose quite high standards of evidence for doing so, as we can have strong incentives (and unconscious biases!) to do it for political reasons.
These are key points that argue against changing the concept boundary to include all conscious reporting of unconscious bias, and more generally push back against unprincipled additions to the concept boundary.
This is not an argument against expanding the concept to include a specific set of phenomena that share the key similarities with the original set, as long as the expansion does not explode the set. I think there may be some things like that within the category of ‘unconscious bias’.
While it is the case that it’s very helpful to have a word for when a human consciously deceives another human, my sense is that there are some important edge cases that we would still call lying, or at least a severe breach of integrity that should be treated similarly to deliberate conscious lies.
Humans are incentivised to self-deceive in the social domain in order to be able to tell convincing lies. It’s sometimes important that if it’s found out that someone strategically self-deceived, that they be punished in some way.
A central example here might be a guy who says he wants to be in a long and loving committed relationship, only to break up after he is bored of the sex after 6-12 months, and really this was predictable from the start if he hadn’t felt it was fine to make big commitments things without introspecting carefully on their truth. I can imagine the woman in this scenario feeling genuinely shocked and lied to. “Hold on, what are you talking about that you feel you want to move out? I am organising my whole life around this relationship, what you are doing right now is calling into question the basic assumptions that you have promised to me.” I can imagine this guy getting a reputation for being untrustworthy and lying to women. I think it is an open question about whether it is accurate for the woman cheated by this man to tell other people that he “lied to her”, though I think it is plausible that I want to punish this behaviour in a similar way that I want to punish much more conscious deception, in a way that motivates me to want to refer to it with the same handle—because it gives you basically very similar operational beliefs about the situation (the person systematically deceived me in a way that was clearly for their personal gain and this hurt me and I think they should be actively punished).
I think I can probably come up with an example where a politician wants power and does whatever is required to take it, such that they end up not being in alignment with the values they stated they held earlier in their career, and allow the meaning of words to fluctuate around them in accordance with what the people giving the politician votes and power want that they end up saying something that is effectively a lie, but that they don’t care about or really notice. This one is a bit slippery for me to point to.
Another context that is relevant: I can imagine going to a scientific conference in a field that has been hit so hard by the replication crisis, that basically all the claims in the conference were false, and I could know this. Not only are the claims at this conference false, but the whole subfield has never been about anything real (example, example, and of course, example). I can imagine a friend of mine attending such a conference and talking to me afterwards, and them thinking that some of the claims seemed true. And I can imagine saying to them “No. You need to understand that all the claims in there are lies. There is no truth-tracking process occurring. It is a sham field, and those people should not be getting funding for their research.” Now, do I think the individuals in the field are immoral? Not exactly, but sorta. They didn’t care about truth and yet paraded themselves as scientists. But I guess that’s a big enough thing in society that they weren’t unusually bad or anything. While it’s not a central case of lying, it currently feels to me like it’s actively helpful for me to use the phrase ‘lie’ and ‘sham’. There is a systematic distortion of truth that gives people resources they want instead of those resources going to projects not systematically distorting reality.
(ADDED: OTOH I do think that I have myself in the past been prompted to want to punish people for these kinds of ‘lies’ in ways that isn’t effective. I have felt that people who have committed severe breaches of integrity in the communities I’m part of are bad people and felt very angry at them, but I think that this has often been an inappropriate response. It does share other important similarities with lies though. Probably want to be a bit careful with the usage here and signal that the part of wanting to immediately socially punish them for a thing that they obviously did wrong is not helpful, because they will feel helpless and not that it’s obvious they did something wrong. But it’s important for me internally to model them as something close to lying, for the sanity of my epistemic state, especially when many people in my environment will not know/think the person has breached integrity and will socially encourage me to positively weight their opinions/statements.)
My current guess at the truth: there are classes of human motivations, such as those for sex, and for prestigious employment positions in the modern world, that have sufficiently systematic biases in favour of self-deception that it is not damaging to add them to the category of ‘lie’ - adding them is not the same as a rule that admits all unconscious bias consciously reported, just a subset that reliably turns up again and again. I think Jessica Taylor / Ben Hoffman / Michael Arc want to use the word ‘fraud’ to refer to it, I’m not sure.
I will actually clean this up and into a post sometime soon. For now let me add another quick hypothesis on this topic whilst crashing from jet lag.
A friend of mine proposed that instead of saying ‘lies’ I could say ‘falsehoods’. Not “that claim is a lie” but “that claim is false”.
I responded that ‘falsehood’ doesn’t capture the fact that you should expect systematic deviations from the truth. I’m not saying this particular parapsychology claim is false. I’m saying it is false in a way where you should no longer trust the other claims, and expect they’ve been optimised to be persuasive.
They gave another proposal, which is to say instead of “they’re lying” to say “they’re not truth-tracking”. Suggest that their reasoning process (perhaps in one particular domain) does not track truth.
I responded that while this was better, it still seems to me that people won’t have an informal understanding of how to use this information. (Are you saying that the ideas aren’t especially well-evidenced? But they sound pretty plausible to me, so let’s keep discussing them and look for more evidence.) There’s a thing where if you say someone is a liar, not only do you not trust them, but you recognise that you shouldn’t even privilege the hypotheses that they produce. If there’s no strong evidence either way, if it turns out the person who told it you is a rotten liar, then if you wouldn’t have considered it before they raised it, don’t consider it now.
Then I realised Jacob had written about this topic a few months back. People talk as though ‘responding to economic incentives’ requires conscious motivation, but actually there are lots of ways that incentives cause things to happen that don’t require humans consciously noticing the incentives and deliberately changing their behaviour. Selection effects, reinforcement learning, and memetic evolution.
Similarly, what I’m looking for is basic terminology for pointing to processes that systematically produce persuasive things that aren’t true, that doesn’t move through “this person is consciously deceiving me”. The scientists pushing adult neurogenesis aren’t lying. There’s a different force happening here that we need to learn to give epistemic weight to the same way we treat a liar, but without expecting conscious motivation to be the root of the force and thus trying to treat it that way (e.g. by social punishment).
More broadly, it seems like there are persuasive systems in the environment that weren’t in the evolutionary environment for adaptation, that we have not collectively learned to model clearly. Perhaps we should invest in some basic terminology that points to these systems so we can learn to not-trust them without bringing in social punishment norms.
The definitional boundaries of “abuser,” as Scott notes, are in large part about coordinating around whom to censure. The definition is pragmatic rather than objective.*
If the motive for the definition of “lies” is similar, then a proposal to define only conscious deception as lying is therefore a proposal to censure people who defend themselves against coercion while privately maintaining coherent beliefs, but not those who defend themselves against coercion by simply failing to maintain coherent beliefs in the first place. (For more on this, see Nightmare of the Perfectly Principled.) This amounts to waging war against the mind.
Of course, in matter of actual fact we don’t strongly censure all cases of consciously deceiving. In some cases (e.g. “white lies”) we punish those who fail to lie, and those who call out the lie. I’m also pretty sure we don’t actually distinguish between conscious deception and e.g. reflexively saying an expedient thing, when it’s abundantly clear that one knows very well that the expedient thing to say is false, as Jessica pointed out here.
*It’s not clear to me that this is a good kind of concept to have, even for “abuser.” It seems to systematically force responses to harmful behavior to bifurcate into “this is normal and fine” and “this person must be expelled from the tribe,” with little room for judgments like “this seems like an important thing for future partners to be warned about, but not relevant in other contexts.” This bifurcation makes me less willing to disclose adverse info about people publicly—there are prominent members of the Bay Area Rationalist community doing deeply shitty, harmful things that I actually don’t feel okay talking about beyond close friends because I expect people like Scott to try to enforce splitting behavior.
Note: I just wrote this in one pass when severely jet lagged, and did not have the effort to edit it much. If I end up turning this into a blogpost I will probably do that. Anyway, I am interested in hearing via PM from anyone who feels that it was sufficiently unclearly written that they had a hard time understanding/reading it.
I recently circled for the first time. I had two one-hour sessions on consecutive days, with 6 and 8 people respectively.
My main thoughts: this seems like a great way for getting to know my acquaintances, connecting emotionally, and build closer relationships with friends. The background emotional processing happening in individuals is repeatedly brought forward as the object of conversation, for significantly enhanced communication/understanding. I appreciated getting to poke and actually find out whether people’s emotional states matched the words they were using. I got to ask questions like:
When you say you feel gratitude, do you just mean you agree with what I said, or do you mean you’re actually feeling warmth toward me? Where in your body do you feel it, and what is it like?
Not that a lot of my circling time was skeptical of people’s words, a lot of the time I trusted the people involved to be accurately reporting their experiences. It was just very interesting—when I noticed I didn’t feel like someone was honest about some micro-emotion—to have the affordance to stop and request an honest internal report.
It felt like there was a constant tradeoff between social-interaction and noticing my internal state. If all I’m doing is noticing my internal state, then I stop engaging with the social environment and don’t have anything off substance to report on. If I just focus on the social interactions, then I never stop and communicate more deeply about what’s happening for me internally. I kept on having an experience like “Hey, I want to interject to add nuance to what you said-” and then stopping and going “So, when you said <x> I felt a sense of irritation/excitement/distrust/other because <y>”.
One moment that I liked a lot, was around the epistemic skill of not deciding your position a second earlier than necessary. Person A was speaking, and person B jumped in and said something that sounded weirdly aggressive. It didn’t make sense, and then person B said “Wait, let me try to figure out what I mean, I feel I’m not using quite the right words”. My experience was first to feel some worry for person A feeling attacked. I quickly calmed down, noticing how thoroughly out of character it would be for person B to actually be saying anything aggressive. I then realised I had a clear hypothesis for what person B actually wanted to say, and waited politely for them to say it. But then I noticed that actually I didn’t have much evidence for my hypothesis at all… so I moved into a state of only curiosity about what person B was going to say, not holding onto my theory of what they would say. And indeed, it turned out they said something entirely different. (I subsequently related this whole experience to person B during the circle.)
This is really important. Being able to hold off on keeping your favoured theory in the back of your head and counting all evidence as pro- or anti- the theory, and instead keeping the theory separate from your identity and feeling full creative freedom to draw a new theory around the evidence that comes in.
There were other personal moments where I brought up how I was feeling toward my friends and they to me, in ways that allowed me to look at long-term connections and short-term conflicts in a clearer light. It was intense.
Both circles were very emotionally interesting and introspectively clarifying, and I will do more with friends in the future.
Reviews of books and films from my week with Jacob:
The Big Short
Review: Really fun. I liked certain elements of how it displays bad nash equilibria in finance (I love the scene with the woman from the ratings agency—it turns out she’s just making the best of her incentives too!).
Review: Wow. A simple story, yet entirely lacking in cliche, and so seemingly original. No cliched characters, no cliched plot twists, no cliched humour, all entirely sincere and meaningful. Didn’t really notice that it was animated (while fantastical, it never really breaks the illusion of reality for me). The few parts that made me laugh, made me laugh harder than I have in ages.
There’s a small visual scene, unacknowledged by the ongoing dialogue, between the mouse-baby and the dust-sprites which is the funniest thing I’ve seen in ages, and I had to rewind for Jacob to notice it.
I liked how by the end, the team of characters are all a different order of magnitude in size.
A delightful, well-told story.
Stranger Than Fiction
Review: This is now my go-to film of someone trying something original and just failing. Filled with new ideas, but none executed well, and overall just a flop, and it really phones-it-in for the last 20 minutes. It does make me notice the distinct lack of originality in most other films that I’ve seen though—most don’t even try to be original like this does. B+ for effort, but D for output.
I Love You, Daddy
Review: A great study of fatherhood, coming of age, and honesty. This was my second watch, and I found many new things that I didn’t find the first time -about what it means to grow up and have responsibility. One moment I absolutely loved is when the Charlie Day character (who was in my opinion representing the id), was brutally honest and totally rewarded for it. I might send this one to my mum, I think she’ll get a lot out of it.
My Dinner With Andre
Review: Very thought-provoking. Jacob and I discussed it for a while afterward. I hope to watch it again some day. I spent 25% of the movie thinking about my own response to what was being discussed, 25% imagining how I would create my version of this film (what the content of the conversation would be), and 50% actually paying close attention to the film.
Overall I felt that both characters were good representatives of their positions, and I liked how much they stuck to their observations over their theories (they talked about what they’d seen and experienced more than they made leaky abstractions and focused on those). The main variable that was not discussed, was technology. It is the agricultural and industrial revolutions that lead to humans feeling so out-of-sorts in the present day world, not any simple fact of how we socialise today, that can simply be fixed / gotten out of. Nonetheless, I expect that the algorithm that Andre is running will help him gain useful insights about how to behave in the modern world. But you do have to actually interface with it and be part of it to have a real cup of coffee waiting for you in the morning, or to lift millions out of poverty.
The last line of Roger Ebert’s review of this was great. Something like: “They’re both trying to get the other to wake up and smell the coffee. Only in Willy’s case, it’s real coffee.”
Books read (only parts of):
Computability and Logic (3rd edition)
I always forget basic definitions of languages and models, so a bunch of time was spent doing that. Jacob and I read half of the chapter on the non-standard numbers, to see how the constructions worked, and I just have the basics down more clearly now. Eliezer’s writings about these numbers connects more strongly to my other learning about first order logic now.
Book is super readable given the subject matter, easy to reference the concepts back to other parts of the book, and all round excellent (though it was the hardest slog on this list). Look forward to reading some more.
Modern Principles: Microeconomics (by Cowen and Tabarrok)
I’ve never read much about supply and demand curves, so it was great to go over them in detail, and how the price equilibrium is reached. We resolved many confusions, that I might end up writing in a LW post. I especially liked learning how the equilibrium price maximises social good, but is not the maximum for either the supplier or the buyer.
It was very wordy and I’d like to read a textbook that had the goal of this level of intuitiveness, but aimed at readers with assumed strong math background. I don’t need paragraphs explaining how to read a 2-D graph each time one comes up.
Jacob made a good point about how the book failed to distinguish hypothesis versus empirical evidence, when presenting standard microeconomic theory. Just because you have the theory down doesn’t mean you should believe it corresponds to reality, but the book didn’t seem to notice the difference.
Overall pretty good. I don’t expect to read most chapters in this book, but we also looked through asymmetric information (some of which later tied into our watching of The Big Short), and there were a few others that looked exciting.
I am in love with this book. I remember picking it up when I was about 17 and not being able to handle it at all and just flicking through to the answers—but this time, especially with Jacob, we were both able to notice when we felt we really understood something and wanted to check the answer to confirm, versus when we’d said ‘reasonable’ things but which didn’t really bottom out in our experiences of the world.
“Well, if you draw the force vectors like this, there should be a normal force of this strength, which splits up into these two basis vectors and so the ball should roll down at this speed.” “Why do you get to assume a force along the normal?” “I don’t know.” “Why do you get to break it up into two vectors who sum to the initial vector?” “I don’t know.” “Then I think we haven’t answered the question yet. Let’s think some more about our experience of balls rolling down hills.”
One of the best things about doing it with Jacob was that I often had cached answers to problems (both from studying mechanics in high school and having read the book 4 years ago), but instead on reading a problem I would give Jacob time to get confused about it, perhaps by supplying useful questions. Then eventually I’d propose my “Well isn’t it obviously X” answer, and Jacob would be able to point out the parts I hadn’t justified from first principles, helping me notice them. There’s a problem in discussing difficult ideas where if people have been taught the passwords, and especially if the passwords have a certain amount of structure that feels like understanding, that it’s hard to notice the gaps. Jacob helped me notice those, and then I could later come up with real answers, that were correct for the right reasons.
The least good thing about this book is the answers to the problems. Often Jacob and I would come up with an answer, then scrap it and build up a first-principles model that predicted it based in our experiences that we were very confident in, and then also deconstruct the initial false intuition some. Then we’d check the answer, and we were right, but the answer didn’t really address the intuitions in either direction, just gave a (correct) argument for the (correct) solution.
I think it might be really valuable to fully deconstruct the intuition behind why people expect a heavier object to fall faster. I’ve made some progress, but it feels like this is a neglected problem of learning a new field—explaining not only what intuitions you should have, but understanding why you assumed something different.
But the value of the book isn’t the answers—it’s the problems. I’ve never experienced such a coherent set of problems, where you can solve each from first principles (and building off what you’ve learned from the previous problems). With most good books, the more you put in the more you get out, but never have I seen a book where you can get this much out of it by putting so much in (most books normally hit a plateau earlier than this one).
Anyway, we got maybe 1/10th through the book. I can’t wait to work through this more the next time I see Jacob.
It’s already affected our discussions of other topics, how well we notice what we do and don’t understand, and what sorts of explanations we look for.
I’m also tempted, for other things I study, to spend less time writing up the insights and instead spend that time coming up with a problem set that you can solve from first principles.
This book made me think that the natural state of learning isn’t ‘reading’ but ‘play’. Playing with ideas, equations, problems, rather than reading and checking understanding.
Jacob and I now have a ritual of continuing the tradition of trying to understand the world, by going to places in Oxford where great thinkers have learned about the universe, and then solving a problem in this book. We visited a square in Magdalen College where Schroedinger worked on his great works, and solved some problems there.
You only get to read this book once. Use it well.
Hanging out with Jacob:
Grade: A++, would do again in a heartbeat.
I talked with Ray for an hour about Ray’s phrase “Keep your beliefs cruxy and your frames explicit”.
I focused mostly on the ‘keep your frames explicit’ part. Ray gave a toy example of someone attempting to communicate something deeply emotional/intuitive, or perhaps a buddhist approach to the world, and how difficult it is to do this with simple explicit language. It often instead requires the other person to go off and seek certain experiences, or practise inhabiting those experiences (e.g. doing a little meditation, or getting in touch with their emotion of anger).
Ray’s motivation was that people often have these very different frames or approaches, but don’t recognise this fact, and end up believing aggressive things about the other person e.g. “I guess they’re just dumb” or “I guess they just don’t care about other people”.
I asked for examples that were motivating his belief—where it would be much better if the disagreers took to hear the recommendation to make their frames explicit. He came up with two concrete examples:
Jim v Ray on norms for shortform, where during one hour they worked through the same reasons-for-disagreement three times.
[blank] v Ruby on how much effort required to send non-threatening signals during disagreements, where it felt like a fundamental value disagreement that they didn’t know how to bridge.
I didn’t get a strong sense for what Ray was pointing at. I see the ways that the above disagreements went wrong, where people were perhaps talking past each other / on the wrong level of the debate, and should’ve done something different. My understanding of Ray’s advice is for the two disagreers to bring their fundamental value disagreements to the explicit level, and that both disagreers should be responsible for making their core value judgements explicit. I think this is too much of a burden to give people. Most of the reasons for my beliefs are heavily implicit and I cannot make things fully explicit ahead of time. In fact, this just seems not how humans work.
One of the key insights that Kahneman’s System 1 and System 2 distinction makes is that my conscious, deliberative thinking (System 2) is a very small fraction of the work my brain is doing, even though it is the part I have the most direct access to. Most of my world-model and decision-making apparatus is in my System 1. There is an important sense in which asking me to make all of my reasoning accessible to my conscious, deliberative system is an AGI-complete request.
What in fact seems sensible to me is that during a conversation I will have a fast feedback-loop with my interlocutor, which will give me a lot of evidence about which part of my thinking to zoom in on and do the costly work of making conscious and explicit. There is great skill involved in doing this live in conversation effectively and repeatedly, and I am excited to read a LW post giving some advice like this.
That said, I also think that many people have good reasons to distrust bringing their disagreements to the explicit level, and rightfully expect it to destroy ability to communicate. I’m thinking of Scott’s epistemic learned helplessness here, but I’m also thinking about experiences where trying to crystalise and name a thought I’m having before I know how to fully express it has a negative effect on my ability to think clearly about it. I’m not sure what this is but this is another time when I feel hesitant to make everything explicit.
As a third thing, my implicit brain is better than my explicit reasoning at modelling social/political dynamics. Let me handwave at a story of a nerd attempting to negotiate with a socially-savvy bully/psychopath/person-who-just-has-different-goals, where the nerd tries to repeatedly and helpfully make all of their thinking explicit, and is confused why they’re losing at the negotiation. I think even healthy and normal people have patterns around disagreement and conflict resolution that could take advantage of a socially inept individually trying to only rely on the things they can make explicit.
These three reasons lead me to not want to advise people to ‘keep their frames explicit’: it seems prohibitively computationally costly to do it for all things, many people should not trust their explicit reasoning to capture their implicit reasons, and that this is especially true for social/political reasoning.
My general impression of this advice is that it seems to want to make everything explicit all of the time (a) as though that were a primitive operation that can solve all problems and (b) I have a sense that it takes up too much of my working memory when I talk with Ray. I have some sense that this approach implies a severe lack of trust in people’s implicit/unconscious reasoning and only believes explicit/conscious reasoning can ever be relied upon, though that seems a bit of a simplistic narrative. (Of course there are indeed reasons to strongly trust conscious reasoning over unconscious—one cannot unconsciously build rockets that fly to the moon—but I think humans do not have the choice to not build a high-trust relationship with their unconscious mind.)
I’d been working on a sequence explaining this all in more detail (I think there’s a lot of moving parts and inferential distance to cover here). I’ll mostly respond in the form of “finish that sequence.”
But here’s a quick paragraph that more fully expands what I actually believe:
If you’re building a product with someone (metaphorical product or literal product), and you find yourself disagreeing, and you explain “This is important because X, which implies Y”, and they say “What!? But, A, therefore B!” and then you both keep repeating those points over and over… you’re going to waste a lot of time, and possibly build a confused frankenstein product that’s less effective than if you could figure out how to successfully communicate.
In that situation, I claim you should be doing something different, if you want to build a product that’s actually good.
If you’re not building a product, this is less obviously important. If you’re just arguing for fun, I dunno, keep at it I guess.
A separate, further claim is that the reason you’re miscommunicating is because you have a bunch of hidden assumptions in your belief-network, or the frames that underly your belief network. I think you will continue to disagree and waste effort until you figure out how to make those hidden assumptions explicit.
You don’t have to rush that process. Take your time to mull over your beliefs, do focusing or whatever helps you tease out the hidden assumptions without accidentally crystallizing them wrong.
Meanwhile, you can reference the fact that the differing assumptions exist by giving them placeholder names like “the sparkly pink purple ball thing”.
This isn’t an “obligation” I think people should have. But I think it’s a law-of-the-universe that if you don’t do this, your group will waste time and/or your product will be worse.
(Lots of companies successfully build products without dealing with this, so I’m not at all claiming you’ll fail. And meanwhile there’s lots of other tradeoffs your company might be making that are bad and should be improved, and I’m not confident this is the most important thing to be working on)
But among rationalists, who are trying to improve their rationality while building products together, I think resolving this issue should be a high priority, which will pay for itself pretty quickly.
Thirdly: I claim there is a skill to building up a model of your beliefs, and your cruxes for those beliefs, and the frames that underly your beliefs… such that you can make normally implicit things explicit in advance. (Or, at least, every time you disagree with someone about one of your beliefs, you automatically flag what the crux for the belief was, and then keep track of it for future reference). So, by the time you get to a heated disagreement, you already have some sense of what sort of things would change your mind, and why you formed the beliefs you did.
You don’t have to share this with others, esp. if they seem to be adversarial. But understanding it for yourself can still help you make sense of the conversation.
Relatedly, there’s a skill to detecting when other people are in a different frame from you, and helping them to articulate their frame.
Literal companies building literal products can alleviate this problem by only hiring people with similar frames and beliefs, so they have an easier time communicating. But, it’s
This seems important because weird, intractable conversations have shown up repeatedly...
in the EA ecosystem
(where even though people are mostly building different products, there is a shared commons that is something of a “collectively built product” that everyone has a stake in, and where billions of dollars and billions of dollars worth of reputation are at stake)
on LessWrong the website
(where everyone has a stake in a shared product of “how we have conversations together” and what truthseeking means)
on the LessWrong development team
where we are literally building a product (a website), and often have persistent, intractable disagreements about UI, minimalism, how shortform should work, is Vulcan a terribly shitshow of a framework that should be scrapped, etc.
every time you disagree with someone about one of your beliefs, you [can] automatically flag what the crux for the belief was
This is the bit that is computationally intractable.
Looking for cruxes is a healthy move, exposing the moving parts of your beliefs in a way that can lead to you learning important new info.
However, there are an incredible number of cruxes for any given belief. If I think that a hypothetical project should accelerate it’s development time 2x in the coming month, I could change my mind if I learn some important fact about the long-term improvements of spending the month refactoring the entire codebase; I could change my mind if I learn that the current time we spend on things is required for models of the code to propagate and become common knowledge in the staff; I could change my mind if my models of geopolitical events suggest that our industry is going to tank next week and we should get out immediately.
I’m not claiming you can literally do this all the time. [Ah, an earlier draft of the previous comment emphasized this this was all “things worth pushing for on the margin”, and explicitly not something you were supposed to sacrifice all other priorities for. I think I then rewrote the post and forgot to emphasize that clarification]
I’ll try to write up better instructions/explanations later, but to give a rough idea of the amount of work I’m talking about. I’m saying “spend a bit more time than you normally do in ‘doublecrux mode’”. [This can be, like, an extra half hour sometimes when having a particular difficult conversation].
When someone seems obviously wrong, or you seem obviously right, ask yourself “what are cruxes are most loadbearing”, and then:
Be mindful as you do it, to notice what mental motions you’re actually performing that help. Basically, do Tuning Your Cognitive Strategies to the double crux process, to improve your feedback loop.
When you’re done, cache the results. Maybe by writing it down, or maybe just sort of thinking harder about it so you remember it a better.
The point is not to have fully mapped out cruxes of all your beliefs. The point is that you generally have practiced the skill of noticing what the most important cruxes are, so that a) you can do it easily, and b) you keep the results computed for later.
Hypothesis: power (status within military, government, academia, etc) is more obviously real to humans, and it takes a lot of work to build detailed, abstract models of anything other than this that feel as real. As a result people who have a basic understanding of a deep problem will consistently attempt to manoeuvre into powerful positions vaguely related to the problem, rather than directly solve the open problem. This will often get defended with “But even if we get a solution, how will we implement it?” without noticing that (a) there is no real effort by anyone else to solve the problem and (b) the more well-understood a problem is, the easier it is to implement a solution.
I think this is true for people who’ve been through a modern school system, but probably not a human universal.
My, that was a long and difficult but worthwhile post. I see why you think it is not the natural state of affairs. Will think some more on it (though can’t promise a full response, it’s quite an effortful post). Am not sure I fully agree with your conclusions.
Seems related to Causal vs Social Reality.
I think of myself as pretty skilled and nuanced at introspection, and being able to make my implicit cognition explicit.
However, there is one fact about me that makes me doubt this severely, which is that I have never ever ever noticed any effect from taking caffeine.
I’ve never drunk coffee, though in the past two years my housemates have kept a lot of caffeine around in the form of energy drinks, and I drink them for the taste. I’ll drink them any time of the day (9pm is fine). At some point someone seemed shocked that I was about to drink one after 4pm, and I felt like I should feel bad or something, so I stopped. I’ve not been aware of any effects.
But two days ago, I finally noticed. I had to do some incredibly important drudge work, and I had two red bulls around 12-2pm. I finished work at 10pm. I realised that while I had not felt weird in any way, I had also not had any of the normal effects of hanging around for hours, which is getting tired, distracted, needing to walk around, wanting to do something different. I had a normal day for 10 hours solely doing crappy things I normally hate.
So I guess now I see the effect of caffeine: it’s not a positive effect, it just removes the normal negative effects of the day. (Which is awesome.)