Some thoughts on Buddhist epistemology.
This risks being threatening, upsetting, and heretical within a certain point of view I commonly see expressed on LW for reasons that will become clear if you keep reading. I don’t know if that means you shouldn’t read this if that sounds like the kind of thing you don’t want to read, but I put it out there so you can make the choice without having to engage in the specifics if you don’t want to. I don’t think you will be missing out on anything if that warning gives you a tinge of “maybe I won’t like reading this”.
My mind produces a type error when people try to perform deep and precise epistemic analysis of the dharma. That is, when they try to evaluate the truth of claims made by the dharma this seems generally fine, but when they go deep enough that they end up trying to evaluate whether the dharma itself is based on something true, I get the type error.
I’m not sure what people trying to do this turn up. My expectation is that their results looks like noise if you aggregate over all such attempts. The reason being that the dharma is not founded on episteme.
As a quick reminder, there are at least three categories of knowledge worth considering: doxa, episteme, and gnosis. Doxa might translate as “hearsay” in English; it’s about statements of the truth. Episteme is knowledge you come to believe via evaluation of the truth. Gnosis is direct, unmediated-by-ontology knowledge of reality. To this I’ll also distinguish techne from episteme, the former being experienced knowledge and the latter being reasoned knowledge.
I’ll make the probably not very bold claim that most LW rationalists value episteme above all else, accept techne as evidence, accept doxa as evidence about evidence and only weak evidence of truth itself, and mostly ignore gnosis because it is not “rational” in the sense that it cannot be put into words and it can only be pointed at by words and so cannot be analyzed because there is no ontology or categorization to allow making claims one way or the other about it.
Buddhist philosophy values gnosis above all else, then techne, then doxa, then episteme.
To say a little more, the most important thing in Buddhist thinking is seeing reality just as it is, unmediated by the “thinking” mind, by which we really mean the acts of discrimination, judgement, categorization, and ontology. To be sure, this “reality” is not external reality, which we never get to see directly, but rather our unmediated contact with it via the senses. But for all the value of gnosis, unless you plan to sit on a lotus flower in perfect equanimity forever and never act in the world, it’s not enough. Techne is the knowledge we gain through action in the world, and although it does pass judgement and discriminate, it also stays close to the ground and makes few claims. It is deeply embodied in action itself.
I’d say doxa comes next because there is a tradition of passing on the words of enlightened people as they said them and acting, at least some of the time, as if they were 100% true. Don’t confuse this for just letting anything in, though: the point is to trust in the words of those who have come before and seen more than you and doing that is often very helpful to learning to see that which was previously invisible for yourself, but it is always an action you do yourself not contingent on the teachings since those only pointed you towards where to look and always failed to put into words (because it was impossible) what you would find. The old story was that the Buddha, when asked why he should be believed, said don’t: try it for yourself and see what you find.
Episteme is last, and that’s because it’s not to be trusted. Of all the ways of knowing, episteme is the least grounded in reality. This should not be surprising, but it might be, so I’ll say a bit about it. Formal methods are not grounded. There’s a reason the grounding problem, epistemic circularity, the problem of the criterion, the problem of finding the universal prior, etc. remain fundamentally unsolved: they are unsolvable in a complete and adequate way. Instead we get pragmatic solutions that cross the chasm between reality and belief, between noumena and phenomena, between the ontic and ontology, and this leap of faith means episteme is always contingent on that leap. Even as it proves things we verify by other means, we must be careful because it’s not grounded and we have to check everything it produces by other means. This means going all the way down to gnosis if possible, and techne at the least.
None of this it to say that episteme is not useful for many things and making predictions, but we hold it at arms length because of its powerful ability to confuse us if we didn’t happen to make the right leaps where we pragmatically had to. It also always leaves something out because it requires distinctions to function, so it is always less complete. At the same time, it often makes predictions that turn out to be true, and the world is better for our powerful application of it. We just have to keep in mind what it is and what it can do and what its dangers are and engage with it in a thoughtful, careful way to avoid getting lost and confusing our perception of reality for reality itself.
So when we talk about the dharma or justify our actions on it, it’s worth noting that it is not really trying to provide consistent episteme. It’s grounded on gnosis and techne, presented via doxa, and only after the fact might we try to extend it via episteme to get an idea of where to look to understand it better. Thus it’s a strange inversion to ask the dharma for episteme-based proofs. It can’t give them, nor does it try, because its episteme is not consistent and cannot be because it chooses completeness instead.
So where does this leave us? If you want to evaluate the dharma, you’ll have to do it yourself. You can’t argue about it or reason it, you have to sit down and look at the nature of reality without conceptualizing it. Maybe that means you won’t engage with it since it doesn’t choose to accept the framing of episteme. That seems fine if you are so inclined. But then don’t be surprised if the dharma claims you are closed minded, if you feel like it attacks your identity, and if it feels just true enough that you can’t easily dismiss it out of hand although you might like to.
So when we talk about the dharma or justify our actions on it, it’s worth noting that it is not really trying to provide consistent episteme. [...] Thus it’s a strange inversion to ask the dharma for episteme-based proofs. It can’t give them, nor does it try, because its episteme is not consistent and cannot be because it chooses completeness instead.
In my view, this seems like a clear failing. The fact that the dharma comes from a tradition where this has usually been the case is not an excuse for not trying to fix it.
Yes, the method requires temporarily suspending episteme-based reasoning and engaging with less conceptual forms of seeing. But it can still be justified and explained using episteme-based models; if it could not, there would be little reason to expect that it would be worth engaging with.
This is not just a question of “the dharma has to be able to justify itself”; it’s also a question of leaving out the episteme component leaves the system impoverished, as noted e.g. here:
Recurrent training to attend to the sensate experience moment-by-moment can undermine the capacity to make meaning of experience. (The psychoanalyst Wilfred Bion described this as an ‘attack on linking’, that is, on the meaning-making function of the mind.) When I ask these patients how they are feeling, or what they are thinking, or what’s on their mind, they tend to answer in terms of their sensate experience, which makes it difficult for them to engage in a transformative process of psychological self-understanding.
In important ways, it is not possible to encounter our unconscious – at least in the sense implied by this perspective – through moment-to-moment awareness of our sensate experience. Yes, in meditation we can have the experience of our thoughts bubbling just beneath the surface – what Shinzen Young calls the brain’s pre-processing – but this is not the unconscious that I’m referring to, it, or at least not all of it.
Let me give an example. Suppose that I have just learned that a close friend has died. I’m deeply saddened by this news. Moments later, I spill a cup of coffee on my new pants and become quite angry. Let’s further suppose that, throughout my life, I’ve had difficulty feeling sadness. For reasons related to my personal history, sadness frightens me. In my moment of anger, if I adopt the perspective of awareness of sensate experience, moment-by-moment, then I will have no access to the fact that I am sad. On the contrary, my sensate experience seems to reflect the fact that I am angry. But given what I know about myself, it’s quite reasonable to posit that my anger is a defense against the feeling of sadness, a feeling of which I am unconscious as I am caught up in my anger.
Hmm, I feel like there’s multiple things going on here, but I think it hinges on this:
Different traditions vary on how much to emphasize models and episteme. None of them completely ignore it, though, only seek to keep it within its proper place. It’s not that episteme is useless, only that it is not primary. You of course should include it because it’s part of the world, and to deny it would lead to confusion and suffering. As you note with your first example especially, some people learn to turn off the discriminating mind rather than hold it as object, and they are worse for it because then they can’t engage with it anymore. Turning it off is only something you could safely do if you really had become so enlightened that you had no shadow and would never accumulate any additional shadow, and even then it seems strange from where I stand to do that although maybe it would make sense to me if I were in the position that it were a reasonable and safe option.
So to me this reads like an objection to a position I didn’t mean to take. I mean to say episteme has a place and is useful, it is not taken as primary to understanding, at some points Buddhist episteme will say contradictory things, that’s fine and expected because dharma episteme is normally post hoc rather than ante hoc (though is still expected to be rational right up until it is forced to hit a contradiction), and ante hoc is okay so long as it is then later verified via gnosis or techne.
>unmediated-by-ontology knowledge of reality.
I think this is a confused concept, related to wrong-way-reduction.
I’ve thought about this a bit and I don’t see a way through to what you are thinking that makes you suggest this since I don’t see a reduction happening here, much less one moving towards bundling together confusion that only looks simpler. Can you say a bit more that might make your perspective on this clearer?
In particular, I think under this formulation knowledge and onotology largely refer to the same thing. Which is part of the reason I think this formulation is mistaken. Separately, I think ‘reality’ has too many moving parts to be useful for the role it’s being used for here.
Maybe, although I think there is a not very clear distinction I’m trying to make between knowledge and ontological knowledge, though maybe it’s not coming across, although if it is and you have some particular argument for why, say, there isn’t or can’t be such a meaningful distinction, I’d be interested to hear it.
As for my model of reality having too many moving parts, you’re right, I’m not totally unconfused about everything yet, and it’s the place the remaining confusion lives.
I agree with KaJ Solata and Viliam that episteme is underweighted in Buddhism, but thanks for explicating that world view
the most important thing in Buddhist thinking is seeing reality just as it is, unmediated by the “thinking” mind, by which we really mean the acts of discrimination, judgement, categorization, and ontology. To be sure, this “reality” is not external reality, which we never get to see directly, but rather our unmediated contact with it via the senses.
The “unmediated contact via the senses” can only give you sensual inputs. Everything else contains interpretation. That means, you can only have “gnosis” about things like [red], [warm], etc. Including a lot of interesting stuff about your inner state, of course, but still fundamentally of the type [feeling this], [thinking that], and perhaps some usually-unknown-to-non-Buddhists [X-ing Y], etc.
Poetically speaking, these are the “atoms of experience”. (Some people would probably say “qualia”.) But some interpretation needs to come to build molecules out of these atoms. Without interpretation, you could barely distinguish between a cat and a warm pillow… which IMHO is a bit insufficient for a supposedly supreme knowledge.
It’s even worse than that, ‘raw’ sensory inputs already have ontological commitments. Those priors inform all our interpretations pre-consciously. Agree that the efficiency of various representations in the context of coherent intents is a good lens.
Could you clarify the distinction between techne and gnosis? Is it something like playing around with a hammer and seeing how it works?
It’s not a very firm distinction, but techne is knowledge from doing, so I would consider playing with a hammer a way to develop techne. It certainly overlaps with the concept of gnosis, which is a bit more general and includes knowledge from direct experience that doesn’t involve “doing”, like the kind of knowledge you gain from observing. But the act of observing is a kind of thing you do, so as you see it’s fuzzy, but generally I think of techne as that which involves your body moving.
I am glad for having read this, but can’t formulate my thoughts super clearly. Just have this vague sense that you’re using too many groundless words and not connecting to the few threads of gnosis(?) that other rationalists would have available.
If an organism is a thing that organizes, then a thing that optimizes is an optimism.
I’m sad that postrationality/metarationality has, as a movement, started to collapse on itself in terms of doing the thing it started out doing.
What I have in mind is that initially, say 5+ years ago, postrationality was something of a banner for folks who were already in the rationalist or rationalist-adjacent community, saw some ways in which rationalists were failing at their own project, and tried to work on figuring out how to do those things.
Now, much like postmodernism before it, I see postrationality collapsing from a thing only for people who were already rationalists and wanted to go beyond its limitations of the time to a kind of prerationality that rejects instead of builds on the rationalist project.
This kind of dynamic is pretty common (cf. premodern, modern, and postmodern) but it still sucks. On the other hand, I guess the good side of it is that I see lots of signs that the rationality community is better integrating some of the early postrationalist insights such that it feels like there’s less to push back against in the median rationalist viewpoint.
Yeah, it seems like postrationalists should somehow establish their rationalist pedigree before claiming the post- title. IIRC, Chapman endorsed this somewhere on twitter? But I can’t find it now. Maybe it was a different postrat. Also it was years ago.
Are there any specific articles you could point out as good examples of this? I don’t remember reading anything about “postrationality” for a year or so—I actually kinda forgot they exist—so I am curious what I missed.
I had a weird feeling from the beginning, when it seemed that Chapman—a leader of a local religious group, if I understand it correctly—became the key figure of “doing rationality better”. On the other hand, it’s not like Less Wrong avoided the religious woo completely. Seems like somehow it only became a minor topic here, and maybe more central one among the postrationalists? (Perhaps because other competing topics, such as AI, were missing?)
Also, I suppose that defining yourself in opposition to something is not helpful to actually finding the “middle way”. Which is why it was easier for rationalists to accept the good arguments made by postrationalists, than the other way round.
This is a short post to register my kudos to LWers for being consistently pretty good at helping each other find answers to questions, or at least make progress towards answers. I feel like I’ve used LW numerous times to make progress on work by saying “here’s what I got, here’s where I’m confused, what do you think?”, whether that be through formal question posts or regular posts that are open ended. Some personal examples that come to mind: recent, older, another.
Praise to the LW community!
People often talk of unconditional love, but they implicitly mean unconditional love for or towards someone or something, like a child, parent, or spouse. But this kind of love is by definition conditional because it is love conditioned on the target being identified as a particular thing within the lover’s ontology.
True unconditional love is without condition, and it cannot be directed because to direct is to condition and choose. Unconditional love is love of all, of everything and all of reality even when not understood as a thing.
Such love is rare, so it seems worth pursuing the arduous cultivation of it.
“love” is poorly-defined enough that it always depends on context. Often, “unconditional love” _is_ expected to be conditional on identity, and really should be called “precommitment against abandonment” or “unconditional support”. But neither of those signal the strength of the intent and safety conferred by the relationship very well.
I _really_ like your expansion into non-identity, though. Love for the real state of the universe, and the simultaneous desire to pick better futures and acceptance of whichever future actually obtains is a mindset I strive for.
Love for the real state of the universe, and the simultaneous desire to pick better futures and acceptance of whichever future actually obtains
This is the hidden half of what got me thinking about this: my growing being with the world as it is rather than as I understand it.
I have a blog post upcoming called ‘Unconditional Love Integration Test: Hitler’
I think it’s safe to say that many LW readers don’t feel like spirituality is a big part of their life, yet many (probably most) people do experience a thing that goes by many names—the inner light, Buddha-nature, shunyata, God—and falls under the heading of “spirituality”. If you’re not sure what I’m talking about, I’m pointing to a common human experience you aren’t having.
Only, I don’t think you’re not having it, you just don’t realize you are having those experiences.
One way some people get in touch with this thing, which I like to think of as “the source” and “naturalness” and might describe as the silently illuminated wellspring, is with drugs, especially psychedelics but really any drug that gets you to either reduce activity of the default-mode network or at least notice it’s operation and stop identifying with it (dissociatives may function like this). In this light, I think of drug users as very spiritual people, only they are unfortunately doing it in a way that is often destructive to their bodies and causes headlessness (causes them to fail to perceive reality accurately and so may act out of confusion and ignorance, leading to greater suffering).
Another way some people manage to get in touch with the source is through exercise. They exercise hard enough that their body gives up devoting enough energy to the brain that the default-mode network shuts down and then they get a “high”.
Another way I think many people touch the source is through nostalgia. My theory is that it works like this: when we are young the conceptualizing mind is weak and we see reality as it is more clearly even if it’s in a way that is very ignorant of causality; then we get older and understand causality better via stronger models and more conceptualization and discernment but at the cost of less seeing reality directly and more seeing it through our maps; nostalgia is then a feeling of longing to get back to the source, to get back to seeing reality directly the way we did when we were younger.
There are many other ways people get in touch with ultimate reality to gain gnosis of it. The ones I’ve just described are all “inferior” in the sense that they partially get back to the source in a fragmented way that lets you get only a piece of it. There are “superior” methods, though, maybe the purest (in the sense of having the least extra stuff and the most clear, direct access) of which I consider to be meditation. But however you do it, the human experience of spirituality is all around you and always available, only we’ve failed to notice the many ways we get at it in the modern, secular world by denying the spirituality of these experiences.
The mentality that lies behind a statement like that seems to me to be pretty dangerous. This is isomorphic to “I know better than other people what’s going on in those other people’s heads; I am smarter/wiser/more observant/more honest.”
Sometimes that’s *true.* Let’s not forget that. Sometimes you *are* the most perceptive one in the room.
But I think it’s a good and common standard to be skeptical of (and even hostile toward) such claims (because such claims routinely lead to unjustified and not-backed-by-reality dismissal and belittlement and marginalization of the “blind” by the “seer”), unless they come along with concrete justification:
Here are the observations that led me to claim that all people do in fact experience X, in direct contradiction of individuals claiming otherwise; here’s why I think I’m correct to ignore/erase those people’s experience.
Here are my causal explanations of why and how people would become blindspotted on X, so that it’s not just a blanket assertion and so that people can attempt to falsify my model.
Here are my cruxes surrounding X; here’s what would cause me to update that I was incorrect in the conclusions I was reaching about what’s going on in other people’s heads
Yeah, I think there’s a subtle distinction. While it’s often correct to believe things that you have a hard time communicating explicitly (e.g. most of my actual world model at any given time), the claim that there’s something definitely true but that in-principle I can’t persuade you of and also can’t explain to you, especially when used by a group of people to coordinate around resources, is often functioning as a coordination flag and not as a description of reality.
Just wanted to note that I am thinking about this exchange, hope to chime in at some point. I’m not sure whether I’m on the same page as Ben about it. May take a couple days to have time to respond in full.
Just a quick update: the mod team just chatted a bunch about this thread. There’s a few different things going on.It’ll probably be another day before a mod follows up here.
[Mod note] I thought for a while about how shortform interacts with moderation here. When Ray initially wrote the shortform announcement post, he described the features, goals, and advice for using it, but didn’t mention moderation. Let me follow-up by saying: You’re welcome and encouraged to enforce whatever moderation guidelines you choose to set on shortform, using tools like comment removal, user bans, and such. As a reminder, see the FAQ section on moderation for instructions on how to use the mod tools. Do whatever you want to help you think your thoughts here in shortform and feel comfortable doing so.
Some background thoughts on this: In other places on the internet, being blocked locks you out of the communal conversation, but there are two factors that make it pretty different here. Firstly, banning someone from a post on LW means they can’t reply to the content they’re banned from, but it doesn’t hide your content from them or their content from you. And secondly, everyone here on LessWrong has a common frontpage where the main conversation happens—the shortform is a low-key place and a relatively unimportant part of the conversation. (You can be banned from posts on frontpage, but that action requires meeting high standards not required for shortform bans.) Relatively speaking, shortform is very low-key, and I expect the median post gets 3x-10x fewer views than the median frontpage post. It’s a place for more casual conversation, hopefully leading to the best ideas getting made into posts—indeed we’re working on adding an option to turn shortform posts into blogposts. This is why we never frontpage a user’s shortform feed—they rarely meet frontpage standards, and they’re not supposed to.
Just to mention this thread in particular, Gordon is well within his rights to ban users or remove their comments from his shortform posts if he wishes to, and the LW mod team will back him up when he wants to do that.
Sure, this is short form. I’m not trying very hard to make a complete argument to defend my thoughts, just putting them out there. There is no norm that I need always abide everywhere to present the best (for some notion of best) version of my reasons for things I claim, least of all, I think, in this space as opposed to, say, in a frontpage post. Thus it feels to me a bit out of place to object in this way here, sort of like objecting that my fridge poetry is not very good or my shower singing is off key.
Now, your point is well taken, but I also generally choose to simply not be willing to cross more than a small amount of inferential distance in my writing (mostly because I think slowly and it requires significant time and effort for me to chain back far enough to be clear to successively wider audiences), since I often think of it as leaving breadcrumbs for those who might be nearby rather than leading people a long way towards a conclusion. I trust people to think things through for themselves and agree with me or not as their reason dictates.
Yes, this means I am often quite distanced from easily verifying the most complex models I have, but such seems to be the nature of complex models that I don’t even have complete in my own mind yet, much less complete in a way that I would lay them out precisely such that they could be precisely verified point by point. This perhaps makes me frustratingly inscrutable about my most exciting claims to those with the least similar priors, but I view it as a tradeoff for aiming to better explain more of the world to myself and those much like me at the expense of failing to make those models legible enough for those insufficiently similar to me to verify them.
Maybe my circumstances will change enough that one day I’ll make a much different tradeoff?
This response missed my crux.
What I’m objecting to isn’t the shortform, but the fundamental presumptuousness inherent in declaring that you know better than everyone else what they’re experiencing, *particularly* in the context of spirituality, where you self-describe as more advanced than most people.
To take a group of people (LWers) who largely say “nah, that stuff you’re on is sketchy and fake” and say “aha, actually, I secretly know that you’re in my domain of expertise and don’t even know it!” is a recipe for all sorts of bad stuff. Like, “not only am I *not* on some sketchy fake stuff, I’m actually superior to my naysayers by very virtue of the fact that they don’t recognize what I’m pointing at! Their very objection is evidence that I see more clearly than they do!”
I’m pouring a lot into your words, but the point isn’t that your words carried all that so much as that they COULD carry all that, in a motte-and-bailey sort of way. The way you’re saying stuff opens the door to abuse, both social and epistemic. My objection wasn’t actually a call for you to give more explanation. It was me saying “cut it out,” while at the same time acknowledging that one COULD, in principle, make the same claim in a justified fashion, if they cared to.
Note: what follows responds literally to what you said. I’m suspicious enough that my interpretation is correct that I’ll respond based on it, but I’m open to the possibility this was meant more metaphorically and I’ve misunderstood your intention.
It was me saying “cut it out,”
It was me saying “cut it out,”
Ah, but that’s not up to you, at least not here. You are welcome to dislike what I say, claim or argue that I am dangerous in some way, downvote me, flag my posts, etc. BUT it’s not up to you to enforce a norm here to the best of my knowledge, even if it’s what you would like to do.
Sorry if that is uncharacteristically harsh and direct of me, but if that was your motivation, I think it important to say I don’t recognize you as having the authority to do that in this space, consider it a violation of my commenting guidelines, and will delete future comments that attempt to do the same.
Hey Gordon, let me see if I understand your model of this thread. I’ll write mine and can you tell me if it matches your understanding?
You write a post giving your rough understanding of a commonly discussed topic that many are confused by
Duncan objects to a framing sentence that he claims means “I know better than other people what’s going on in those other people’s heads; I am smarter/wiser/more observant/more honest.” because it seems inappropriate and dangerous in this domain (spirituality)
You say “Dude, I’m just getting some quick thoughts off my chest, and it’s hard to explain everything”
Duncan says you aren’t responding to him properly—he does not believe this is a disagreement but a norm-violation
You say that Duncan is not welcome to prosecute norm violations on your wall unless they are norms that you support
Yes, that matches my own reading of how the interaction progressed, caveat any misunderstanding I have of Duncan’s intent.
nods Then I suppose I feel confused by your final response.
If I imagine writing a shortform post and someone said it was:
Very rude to another member of the community
Endorsing a study that failed to replicate
Lied about an experience of mine
Tried to unfairly change a narrative so that I was given more status
I would often be like “No, you’re wrong” or maybe “I actually stand by it and intended to be rude” or “Thanks, that’s fair, I’ll edit”. I can also imagine times where the commenter is needlessly aggressive and uncooperative where I’d just strong downvote and ignore.
But I’m confused by saying “you’re not allowed to tell me off for norm-violations on my shortform”. To apply that principle more concretely, it could say “you’re not allowed to tell me off for lying on my shortform”.
My actual model of you feels a bit confused by Duncan’s claim or something, and wants to fight back against being attacked for something you don’t see as problematic. Like, it feels presumptuous of Duncan to walk into your post and hold you to what feels mostly like high standards of explanation, and you want to (rightly) say that he’s not allowed to do that.
Does that all seem right?
Yes. To add to this what I’m most strongly reacting to is not what he says he’s doing explicitly, which I’m fine with, but what further conversation suggests he is trying to do: to act as norm enforcer rather than as norm enforcement recommender.
I explicitly reject Gordon’s assertions about my intentions as false, and ask (ASK, not demand) that he justify (i.e. offer cruxes) or withdraw them.
I cannot adequately do that here because it relies on information you conveyed to me in a non-public conversation.
I accept that you say that’s not what you’re doing, and I am happy to concede that your internal experience of yourself as you experience it tells you that you are doing what you are doing, but I now believe that my explanation better describes why you are doing what you are doing than the explanation you are able to generate to explain your own actions.
The best I can maybe offer is that I believe you have said things that are better explained by an intent to enforce norms rather than argue for norms and imply that general case should be applied in this specific case. I would say the main lines of evidence revolve around how I interpret your turns of phrase, how I read your tone (confrontational and defensive), what aspects of things I have said you have chosen to respond to, how you have directed the conversation, and my general model of human psychology with the specifics you are giving me filled in.
Certainly I may be mistaken in this case and I am reasoning off circumstantial evidence which is not a great situation to be in, but you have pushed me hard enough here and elsewhere that it has made me feel it is necessary to act to serve the purpose of supporting the conversation norms I prefer in the places you have engaged me. I would actually really like this conversation to end because it is not serving anything I value, other than that I believe not responding would simply allow what I dislike to continue and be subtly accepted, and I am somewhat enjoying the opportunity to engage in ways I don’t normally so I can benefit from the new experience.
I note for the record that the above is strong evidence that Gordon was not just throwing an offhand turn of phrase in his original post; he does and will regularly decide that he knows better than other people what’s going on in those other people’s heads. The thing I was worried about, and attempting to shine a light on, was not in my imagination; it’s a move that Gordon endorses, on reflection, and it’s the sort of thing that, historically, made the broader culture take forever to recognize e.g. the existence of people without visual imagery, or the existence of episodics, or the existence of bisexuals, or any number of other human experiences that are marginalized by confident projection.
I’m comfortable with just leaving the conversation at “he, I, and LessWrong as a community are all on the same page about the fact that Gordon endorses making this mental move.” Personally, I find it unjustifiable and morally abhorrent. Gordon clearly does not. Maybe that’s the crux.
[He] does and will regularly decide that he knows better than other people what’s going on in those other people’s heads. [...] Personally, I find it unjustifiable and morally abhorrent.
[He] does and will regularly decide that he knows better than other people what’s going on in those other people’s heads. [...] Personally, I find it unjustifiable and morally abhorrent.
How can it be morally abhorrent? It’s an epistemic issue. Factual errors often lead to bad consequences, but that doesn’t make those errors moral errors. A moral error is an error about a moral fact, assignement of value to situations, as opposed to prediction of what’s going on. And what someone thinks is a factual question, not a question of assigning value to an event.
Things that are morally abhorrent are not necessarily moral errors. For example I can find wildlife suffering morally abhorrent but there’s obviously no moral errors or any kind of errors being committed there. Given that the dictionary defines abhorrent as “inspiring disgust and loathing; repugnant” I think “I find X morally abhorrent” just means “my moral system considers X to be very wrong or to have very low value.”
That’s one way for my comment to be wrong, as in “Systematic recurrence of preventable epistemic errors is morally abhorrent.”
When I was writing the comment, I was thinking of another way it’s wrong: given morality vs. axiology distinction, and distinction between belief and disclosure of that belief, it might well be the case that it’s a useful moral principle to avoid declaring beliefs about what others think, especially when those others disagree with the declarations. In that case it’s a violation of this principle, a moral wrong, to declare such beliefs. (A principle like this gets in the way of honesty, so promoting it is contentious and shouldn’t be an implicit background assumption. And the distinction between belief and its declaration was not clearly made in the above discussion.)
I find it morally abhorrent because, when not justified and made-cruxy (i.e. when done the only way I’ve ever seen Gordon do it), it’s tantamount to trying to erase another person/another person’s experience, and (as noted in my first objection) it often leads, in practice, to socially manipulative dismissiveness and marginalization that’s not backed by reality.
So it’s a moral principle under the belief vs. declaration distinction (as in this comment). In that case I mostly object to not making that distinction (a norm to avoid beliefs of that form is on entirely different level than a norm to avoid their declarations).
Personally I don’t think the norm about declarations is on the net a good thing, especially on LW, as it inhibits talking about models of thought. The examples you mentioned are important but should be covered by a more specialized norm that doesn’t cause as much collateral damage.
I’m not sure I’m exactly responding to what you want me to respond to, but:
It seems to me that a declaration like “I think this is true of other people in spite of their claims to the contrary; I’m not even sure if I could justify why? But for right now, that’s just the state of what’s in my head”
is not objectionable/doesn’t trigger the alarm I was trying to raise. Because even though it fails to offer cruxes or detail, it at least signals that it’s not A STATEMENT ABOUT THE TRUE STATE OF THE UNIVERSE, or something? Like, it’s self-aware about being a belief that may or may not match reality?
Which makes me re-evaluate my response to Gordon’s OP and admit that I could have probably offered the word “think” something like 20% more charity, on the same grounds, though on net I still am glad that I spelled out the objection in public (like, the objection now seems to me to apply a little less, but not all the way down to “oops, the objection was fundamentally inappropriate”).
(By “belief” I meant a belief that talkes place in someone’s head, and its existence is not necessarily communicated to anyone else. So an uttered statement “I think X” is a declaration of belief in X, not just a belief in X. A belief in X is just a fact about that person’s mind, without an accompanying declaration. In this framing, the version of the norm about beliefs (as opposed to declarations) is the norm not to think certain thoughts, not a norm to avoid sharing the observations about the fact that you are thinking them.)
I think a salient distinction between declarations of “I think X” and “it’s true that X” is a bad thing, as described in this comment. The distinction is that in the former case you might lack arguments for the belief. But if you don’t endorse the belief, it’s no longer a belief, and “I think X” is a bug in the mind that shouldn’t be called “belief”. If you do endorse it, then “I think X” does mean “X”. It is plausibly a true statement about the state of the universe, you just don’t know why; your mind inscrutably says that it is and you are inclined to believe it, pending further investigation.
So the statement “I think this is true of other people in spite of their claims to the contrary” should mean approximately the same as “This is true of other people in spite of their claims to the contrary”, and a meaningful distinction only appears with actual arguments about those statements, not with different placement of “I think”.
I forget if we’ve talked about this specifically before, but I rarely couch things in ways that make clear I’m talking about what I think rather than what is “true” unless I am pretty uncertain and want to make that really clear or expect my audience to be hostile or primarily made up of essentialists. This is the result of having an epistemology where there is no direct access to reality so I literally cannot say anything that is not a statement about my beliefs about reality, so saying “I think” or “I believe” all the time is redundant because I don’t consider eternal notions of truth meaningful (even mathematical truth, because that truth is contingent on something like the meta-meta-physics of the world and my knowledge of it is still mediated by perception, cf. certain aspects of Tegmark).
I think of “truth” as more like “correct subjective predictions, as measured against (again, subjective) observation”, so when I make claims about reality I’m always making what I think of as claims about my perception of reality since I can say nothing else and don’t worry about appearing to make claims to eternal, essential truth since I so strongly believe such a thing doesn’t exist that I need to be actively reminded that most of humanity thinks otherwise to some extent. Sort of like going so hard in one direction that it looks like I’ve gone in the other because I’ve carved out everything that would have allowed someone to observe me having to navigate between what appear to others to be two different epistemic states where I only have one of them.
This is perhaps a failure of communication, and I think I speak in ways in person that make this much clearer and then I neglect the aspects of tone not adequately carried in text alone (though others can be the judge of that, but I basically never get into discussions about this concern in person, even if I do get into meta discussions about other aspects of epistemology). FWIW, I think Eliezer has (or at least had) a similar norm, though to be fair it got him into a lot of hot water too, so maybe I shouldn’t follow his example here!
leaving the conversation at “he, I, and LessWrong as a community are all on the same page about the fact that Gordon endorses making this mental move.”
leaving the conversation at “he, I, and LessWrong as a community are all on the same page about the fact that Gordon endorses making this mental move.”
Nesov scooped me on the obvious objection, but as long as we’re creating common knowledge, can I get in on this? I would like you and Less Wrong as a community to be on the same page about the fact that I, Zack M. Davis, endorse making the mental move of deciding that I know better than other people what’s going on in those other people’s heads when and only when it is in fact the case that I know better than those other people what’s going on in their heads (in accordance with the Litany of Tarski).
the existence of bisexuals
the existence of bisexuals
As it happens, bisexual arousal patterns in men are surprisingly hard to reproduce in the lab! This is a (small, highly inconclusive) example of the kind of observation that one might use to decide whether or not we live in a world in which the cognitive algorithm of “Don’t decide that you know other people’s minds better than they do” performs better or worse than other inference procedures.
J. Michael Bailey, “What Is Sexual Orientation and Do Women Have One?”, section titled “Sexual Arousal Patterns vs. the Kinsey Scale: The Case of Male Bisexuality” ↩︎
when and only when it is in fact the case that I know better than those other people what’s going on in their heads (in accordance with the Litany of Tarski).
Yes, as clearly noted in my original objection, there is absolutely a time and a place for this, and a way to do it right; I too share this tool when able and willing to justify it. It’s only suspicious when people throw it out solely on the strength of their own dubious authority. My whole objection is that Gordon wasn’t bothering to (I believe as a cover for not being able to).
as clearly noted in my original objection
as clearly noted in my original objection
Acknowledged. (It felt important to react to the great-grandparent as a show of moral resistance to appeal-to-inner-privacy conversation halters, and it was only after posting the comment that I remembered that you had acknolwedged the point earlier in the thread, which, in retrospect, I should have at least acknowledged even if the great-grandparent still seemed worth criticizing.)
there is absolutely a time and a place for this
there is absolutely a time and a place for this
Exactly—and lesswrong.com is the place for people to report on their models of reality, which includes their models of other people’s minds as a special case.
Other places in Society are right to worry about erasure, marginalization, and socially manipulative dismissiveness! But in my rationalist culture, while standing in the Citadel of Truth, we’re not allowed to care whether a map is marginalizing or dismissive; we’re only allowed to care about whether the map reflects the territory. (And if there are other cultures competing for control of the “rationalist” brand name, then my culture is at war with them.)
My whole objection is that Gordon wasn’t bothering to
My whole objection is that Gordon wasn’t bothering to
Great! Thank you for critcizing people who don’t justify their beliefs with adequate evidence and arguments. That’s really useful for everyone reading!
(I believe as a cover for not being able to).
(I believe as a cover for not being able to).
In context, it seems worth noting that this is a claim about Gordon’s mind, and your only evidence for it is absence-of-evidence (you think that if he had more justification, he would be better at showing it). I have no problem with this (as we know, absence of evidence is evidence of absence), but it seems in tension with some of your other claims?
criticizing people who don’t justify their beliefs with adequate evidence and arguments
criticizing people who don’t justify their beliefs with adequate evidence and arguments
I think justification is in the nature of arguments, but not necessary for beliefs or declarations of beliefs. A belief offered without justification is a hypothesis called to attention. It’s concise, and if handled carefully, it can be sufficient for communication. As evidence, it’s a claim about your own state of mind, which holds a lot of inscrutable territory that nonetheless can channel understanding that doesn’t yet lend itself to arguments. Seeking arguments is certainly a good thing, to refactor and convey beliefs, but that’s only a small part of how human intelligence builds its map.
Yeah, if I had the comment to rewrite (I prefer not to edit it at this point) I would say “My whole objection is that Gordon wasn’t bothering to (and at this point in the exchange I have a hypothesis that it’s reflective of not being able to, though that hypothesis comes from gut-level systems and is wrong-until-proven-right as opposed to, like, a confident prior).”
So, having a little more space from all this now, I’ll say that I’m hesitant to try to provide justifications because certain parts of the argument require explaining complex internal models of human minds that are a level more complex than I can explain even though I’m using them (I only seem to be able to interpret myself coherently one level of organization less than the maximum level of organization present in my mind) and because other parts of the argument require gnosis of certain insights that I (and to the best of my knowledge, no one) knows how to readily convey without hundreds to thousands of hours of meditation and one-on-one interactions (though I do know a few people who continue to hope that they may yet discover a way to make that kind of thing scalable even though we haven’t figured it out in 2500 years, maybe because we were missing something important to let us do it).
So it is true that I can’t provide adequate episteme of my claim, and maybe that’s what you’re reacting to. I don’t consider this a problem, but I also recognize that within some parts of the rationalist community that is considered a problem (I model you as being one such person, Duncan). So given that, I can see why from your point of view it looks like I’m just making stuff up or worse since I can’t offer “justified belief” that you’d accept as “justified”, and I’m not really much interested in this particular case in changing your mind as I don’t yet completely know myself how to generate that change in stance towards epistemology in others even though I encountered evidence that lead me to that conclusion myself.
There’s a dynamic here that I think is somewhat important: socially recognized gnosis.
That is, contemporary American society views doctors as knowing things that laypeople don’t know, and views physicists as knowing things that laypeople don’t know, and so on. Suppose a doctor examines a person and says “ah, they have condition X,” and Amy responds with “why do you say that?”, and the doctor responds with “sorry, I don’t think I can generate a short enough explanation that is understandable to you.” It seems like the doctor’s response to Amy is ‘socially justified’, in that the doctor won’t really lose points for referring to a pre-existing distinction between those-in-the-know and laypeople (except maybe for doing it rudely or gracelessly). There’s an important sense in which society understands that it in fact takes many years of focused study to become a physicist, and physicists should not be constrained by ‘immediate public justification’ or something similar.
But then there’s a social question, of how to grant that status. One might imagine that we want astronomers to be able to do their astronomy and have their unintelligibility be respected, while we don’t want to respect the unintelligibility of astrologers.
So far I’ve been talking ‘nationally’ or ‘globally’ but I think a similar question holds locally. Do we want it to be the case that ‘rationalists as a whole’ think that meditators have gnosis and that this is respectable, or do we want ‘rationalists as a whole’ to think that any such respect is provisional or ‘at individual discretion’ or a mistake?
That is, when you say:
I don’t consider this a problem, but I also recognize that within some parts of the rationalist community that is considered a problem (I model you as being one such person, Duncan).
I feel hopeful that we can settle whether or not this is a problem (or at least achieve much more mutual understanding and clarity).
So it is true that I can’t provide adequate episteme of my claim, and maybe that’s what you’re reacting to.
This feels like the more important part (“if you don’t have episteme, why do you believe it?”) but I think there’s a nearly-as-important other half, which is something like “presenting as having respected gnosis” vs. “presenting as having unrespected gnosis.” If you’re like “as a doctor, it is my considered medical opinion that everyone has spirituality”, that’s very different from “look, I can’t justify this and so you should take it with a grain of salt, but I think everyone secretly has spirituality”. I don’t think you’re at the first extreme, but I think Duncan is reacting to signals along that dimension.
That’s not the point! Zack is talking about beliefs, not their declaration, so it’s (hopefully) not the case that there is “a time and a place” for certain beliefs (even when they are not announced), or that beliefs require ability and willingness to justify them (at least for some senses of “justify” and “belief”).
Oh, one last footnote: at no point did I consider the other conversation private, at no point did I request that it be kept private, and at no point did Gordon ask if he could reference it (to which I would have said “of course you can”). i.e. it’s not out of respect for my preferences that that information is not being brought in this thread.
Correct, it was made in a nonpublic but not private conversation, so you are not the only agent to consider, though admittedly the primary one other than myself in this context. I’m not opposed to discussing disclosure, but I’m also happy to let the matter drop at this point since I feel I have adequately pushed back against the behavior I did not want to implicitly endorse via silence since that was my primary purpose in continuing these threads past the initial reply to your comment.
There’s a world of difference between someone saying “[I think it would be better if you] cut it out because I said so” and someone saying “[I think it would be better if you] cut it out because what you’re doing is bad for reasons X, Y, and Z.” I didn’t bother to spell out that context because it was plainly evident in the posts prior. Clearly I don’t have any authority beyond the ability to speak; to
claim or argue that I am dangerous in some way
IS what I was doing, and all I was doing.
I mostly disagree that better reasons matter in a relevant way here, especially since I am currently reading your intent as not one of informing me of that you think there is a norm that should be enforced but instead a bid to enforce that norm. To me what’s relevant is intended effect.
What’s the difference?
Suppose I’m talking with a group of loose acquaintances, and one of them says (in full seriousness), “I’m not homophobic. It’s not that I’m afraid of gays, I just think that they shouldn’t exist.”
It seem to me that it is appropriate for me to say, “Hey man, that’s not ok to say.” It might be that a number of other people in the conversation would back me up (or it might be that they they defend the first guy), but there wasn’t common knowledge of that fact beforehand.
In some sense, this is a bid to establish a new norm, by pushing a the private opinions of a number of people into common knowledge. It also seems to me to be a virtuous thing to do in many situations.
(Noting that my response to the guy is not: “Hey, you can’t do that, because I get to decide what people do around here.” It’s “You can’t do that, because it’s bad” and depending on the group to respond to that claim in one way or another.)
“Here are some things you’re welcome to do, except if you do them I will label them as something else and disagree with them.”
Your claim that you had tentative conclusions that you were willing to update away from is starting to seem like lip service.
I am currently reading your intent as not one of informing me of that you think there is a norm that should be enforced
Literally my first response to you centers around the phrase “I think it’s a good and common standard to be skeptical of (and even hostile toward) such claims.” That’s me saying “I think there’s a norm here that it’s good to follow,” along with detail and nuance à la here’s when it’s good not to follow it.
This is a question of inferred intent, not what you literally said. I am generally hesitant to take much moderation action based on what I infer, but you have given me additional reason to believe my interpretation is correct in a nonpublic thread on Facebook.
(If admins feel this means I should use a reign of terror moderation policy I can switch to that.)
Regardless, I consider this a warning of my local moderation policy only and don’t plan to take action on this particular thread.
Er, I generally have FB blocked, but I have now just seen the thread on FB that Duncan made about you, and that does change how I read the dialogue (it makes Duncan’s comments feel more like they’re motivated by social coordination around you rather than around meditation/spirituality, which I’d previously assumed).
(Just as an aside, I think it would’ve been clearer to me if you’d said “I feel like you’re trying to attack me personally for some reason and so it feels especially difficult to engage in good faith with this particular public accusation of norm-violation” or something like that.)
I may make some small edit to my last comment up-thread a little after taking this into account, though I am still curious about your answer to the question as I initially stated it.
I can have different agendas and follow different norms on different platforms. Just saying. If I were trying to do the exact same thing in this thread as I am in the FB thread, they would have the same words, instead of different words.
(The original objection *does* contain the same words, but Gordon took the conversation in meaningfully different directions on the two different platforms.)
I note that above, Gordon is engaging in *exactly* the same behavior that I was trying to shine a spotlight on (claiming to understand my intent better than I do myself/holding to his model that I intend X despite my direct claims to the contrary).
Outside observer takeaway: There’s a bunch of sniping and fighting here, but if I ignore all the fighting and look at only the ideas, what we have is that Gordon presented an idea, Duncan presented counterarguments, and Gordon declined to address the counterarguments. Posting on shortform doesn’t come with an obligation to follow up and defend things; it’s meant to be a place where tentative and early stage ideas can be thrown around, so that part is fine. But I did come away believing the originally presented idea is probably wrong.
(Some of the meta-level fighting seemed not-fine, but that’s for another comment.)
Seems to me that modern life full of distractions. As a smart person, you probably have a work that requires thinking (not just moving your muscles in a repetitive way). In your free time there is internet with all the websites optimized for addictiveness. Plus all the other things you want to do (books to read, movies to see, friends to visit). Electricity can turn your late night into a day; you can take a book or a smartphone everywhere.
So, unless we choose it consciously, there are no silent moments, to get in contact with yourself… or whatever higher power you imagine there to be, talking to you.
I wonder what is the effect ratio between meditation and simply taking a break and wondering about stuff. Maybe it’s our productivity-focused thinking saying that meditating (doing some hard work in order to gain supernatural powers) is a worthy endeavor, while goofing off is a sin.
“Simply taking a break and wondering about stuff” is a decent way to get in touch with this thing I’m pointing at. The main downside to it is that it’s slow, in that for it to produce effects similar to meditation probably requires an order of magnitude more time, and likely won’t result in the calmest brain states where you can study your phenomenology clearly.
Are there individuals willing to explicitly engage in comforting discussion regarding these things you’ve written about? Any willing to extend personal invitations?
I would love to discuss spirituality with otherwise “rational” intelligent people.
Please consider teaching out to me personally—it would be transformative: email@example.com
I have plans to write this up more fully as a longer post explaining the broader ideas with visuals, but I thought I would highlight one that is pretty interesting and try out the new shortform feature at the same time! As such, this is not optimized for readability, has no links, and I don’t try to backup my claims. You’ve been warned!
Suppose you frequently found yourself identifying with and feeling like you were a homunculus controlling your body and mind: there’s a real you buried inside, and it’s in the driver’s seat. Sometimes your mind and body do what “you” want, sometimes it doesn’t and this is frustrating. Plenty of folks reify this in slightly different ways: rider and elephant, monkey and machine, prisoner in cave (or audience member in theater), and, to a certain extent, variations on the S1/S2 model. In fact, I would propose this is a kind of dual process theory of mind that has you identifying with one of the processes.
A few claims.
First, this is a kind of constant, low-level dissociation. It’s not the kind of high-intensity dissociation we often think of when we use that term, but it’s still a separation of sense of self from the physical embodiment of self.
Second, this is projection, and thus a psychological problem in need of resolving. There’s nothing good about thinking of yourself this way; it’s a confusion that may be temporarily helpful but it’s also something you need to learn to move beyond via first reintegrating the separated sense of self and mind/body.
Third, people drawn to the rationalist community are unusually likely to be the sort of folks who dissociate and identify with the homunculus, S2, the rider, far mode, or whatever you want to call it. It gives them a world view that says “ah, yes, I know what’s right, but for some reason by stupid brain doesn’t do what I want, so let’s learn how to make it do what I want” when this is in fact a confusion because it’s the very brain that’s “stupid” that’s producing the feeling that you think you know what you want!
To speculate a bit, this might help explain some of the rationalist/meta-rationalist divide: rationalists are still dissociating, meta-rationalists have already reintegrated, and as a result we care about very different things and look at the world differently because of it. That’s very speculative, though, and I have nothing other than weak evidence to back it up.
More surprised than perhaps I should be that people take up tags right away after creating them. I created the IFS tag just a few days ago after noticing it didn’t exist but wanted to link it and I added the first ~5 posts that came up if I searched for “internal family systems”. It now has quite a few more posts tagged with it that I didn’t add. Super cool to see the system working in real time!
tl;dr: read multiple things concurrently so you read them “slowly” over multiple days, weeks, months
When I was a kid, it took a long time to read a book. How could it not: I didn’t know all the words, my attention span was shorter, I was more restless, I got lost and had to reread more often, I got bored more easily, and I simply read fewer words per minute. One of the effects of this is that when I read a book I got to live with it for weeks or months as I worked through it.
I think reading like that has advantages. By living with a book for longer the ideas it contained had more opportunity to bump up against other things in my life. I had more time to think about what I had read when I wasn’t reading. I more deeply drunk in the book as I worked to grok it. And for books I read for fun, I got to spend more time enjoying them, living with the characters and author, by having it spread out over time.
As an adult it’s hard to preserve this. I read faster and read more than I did as a kid (I estimate I spend 4 hours a day reading on a typical day (books, blogs, forums, etc.), not including incidental reading in the course of doing other things). Even with my relatively slow reading rate of about 200 wpm, I can polish off ~50k words per day, the length of a short novel.
The trick, I find, is to read slowly by reading multiple things concurrently and reading only a little bit of each every day. For books this is easy: I can just limit myself to a single chapter per day. As long as I have 4 or 5 books I’m working on at once, I can spread out the reading of each to cover about a month. Add in other things like blogs and I can spread things out more.
I think this has additional benefits over just getting to spend more time with the ideas. It lets the ideas in each book come up against each other in ways they might otherwise not. I sometimes notice patterns that I might otherwise not have because things are made simultaneously salient that otherwise would not be. And as a result I think I understand what I read better because I get the chance not just to let it sink in over days but also because I get to let it sink in with other stuff that makes my memory of it richer and more connected.
So my advice, if you’re willing to try it, is to read multiple books, blogs, etc. concurrently, only reading a bit of each one each day, and let your reading span weeks and months so you can soak in what you read more deeply rather than letting it burn bright and fast through your mind to be forgotten like a used up candle.
Interesting idea, thanks. I think this also hints at other ways to approach this (i.e. maybe rather than interspersing books with other books, you could interspersing them with non-reading-things that still give you some chance to have idea from multiple domains bumping into each other)
Explanations are liftings from one ontology to another.
Seems true, although in some cases I feel like one of the ontologies is just an obviously bigger/better version of another one.
This actually fits the lifting metaphor (which is itself a metaphor)!
I get worried about things like this article that showed up on the Partnership on AI blog. Reading it there’s nothing I can really object to in the body of post: it’s mostly about narrow AI alignment and promotes a positive message of targeting things that benefit society rather than narrowly maximize a simple metric. How it’s titled “Aligning AI to Human Values means Picking the Right Metrics” and that implies to me a normative claim that reads in my head something like “to build aligned AI it is necessary and sufficient to pick the right metrics” which is something I think few would agree with. Yet if I was a casual observer just reading the title of this post I might come away with the impression that AI alignment is as easy as just optimizing for something prosocial, not that there are lots of hard problems to be solved to even get AI to do what you want, let alone to pick something beneficial to humanity to do.
To be fair this article has a standard “not necessarily the views of PAI, etc.” disclaimer, but then the author is a research fellow at PAI.
This makes me a bit nervous about the effect of PAI on promoting AI safety in industry, especially if it effectively downplays it or makes it seem easier than it is in ways that either encourages or fails to curtail risky behavior in the use of AI in industry.
Hi Gordon. Thanks for reading the post. I agree completely that the right metrics are nowhere near sufficient for aligned AI — further I’d say that “right” and “aligned” have very complex meanings here.
What I am trying to do with this post is shed some light on one key piece of the puzzle, the actual practice of incorporating metrics into real systems. I believe this is necessary, but don’t mean to suggest that this is sufficient or unproblematic. As I wrote in the post, “this sort of social engineering at scale has all the problems of large AI systems, plus all the problems of public policy interventions.”
To me the issue is that large, influential optimizing systems already exist and seem unlikely to be abandoned. There may be good arguments that a particular system should not be used, but it’s hard for me to see an argument to avoid this category of technology as a whole. As I see it, the question is not so much “should we try to choose appropriate metrics?” but “do we care to quantitatively monitor and manage society-scale optimizing systems?” I believe this is an urgent need for this sort of work within industry.
Having said all that, you may be right that the title of this post overpromises. I’d welcome your thoughts here.
I recently watched all 7 seasons of HBO’s “Silicon Valley” and the final episode (or really the final 4 episodes leading up into the final one) did a really great job of hitting on some important ideas we talk about in AI safety.
Now, the show in earlier seasons has played with the idea of AI with things like an obvious parody of Ben Goertzel and Sophia, discussion of Roko’s Basilisk, and of course AI that Goodharts. In fact, Goodharting is a pivotal plot point in how the show ends, along with a Petrov-esque ending where hard choices have to be made under uncertainty to protect humanity and it has to be kept a secret due to an information hazard.
Goodhart, Petrov, and information hazards are not mentioned by name in the show, but the topics are clearly present. Given that the show was/is popular with folks in the SF Bay Area tech scene because it does such a good job of mirroring back what it’s like to live in that scene, even if it’s a hyperbolic characterization, I wonder if and hope that this will helpfully nudge folks towards normalizing taking AI safety seriously and seeing it as virtuous to forgo personal gain in exchange for safeguarding humanity.
I don’t expect for things to change dramatically because of the show, but on the margin it might be working to make us a little bit safer. For that reason I think it’s likely a good idea to encourage folks not already dedicated to AI safety to watch the show, so long as the effort involved in minimal.
NB: There’s something I feel sad about when I imagine what it’s like to be others, so I’m going to ramble about it a bit in shortform because I’d like to say this and possibly say it confusingly rather than not say it at all. Maybe with some pruning this babble can be made to make sense.
There’s a certain strain of thought and thinkers in the rationality community that make me feel sad when I think about what it must be like to be them: the “closed” individualists. This is as opposed to people who view personal identity as either “empty” or “open”.
I’ll let Andrés of QRI explain all too briefly:
Closed Individualism: You start existing when you are born, and stop when you die.
Empty Individualism: You exist as a “time-slice” or “moment of experience.”
Open Individualism: There is only one subject of experience, who is everyone.
I might summarize the positions a little differently. Closed individualism is the “naive” theory of individualism: people, agents, etc. are like islands forever separated from each other by the gulf of subjective experience that can only be crossed by sending messages in bottles to the other islands (because you can never leave the island you are on). Empty individualism says that individualism is an after the fact reification and is not a natural phenomenon but rather an illusory artifact of how we understand the world. Open individualism is a position like the thing panpsychists are often trying to backpedal from, that the Universe is experiencing itself through us.
I think other positions are possible. For example, my own thinking is that it’s more like seeing these all as partial views that are “right” from a certain frame of thinking but none on its own captures the whole thing. I might call my position something like dialectical empty individualism via comparison to dialectical monism (which I think is the right term to capture my metaphysical position, though neutral monism probably works just as well, ergo neutral empty individualism might be an alternative term).
Anyway, back to the sadness. Now to be fair I feel sad when I think about what it must be like to be anyone who holds tightly to a closed individualism perspective, rationalist or not, but I more often see the extremes of where the closed position takes one among rationalists. I’m making an inference here, but my guess is that a closed individualist view is a large part of what makes things like value drift scary, life extension a top priority, and game & decision theory feel vitally important not just to AI safety but to living life.
And I say all this having previously been a closed individualist for most of my life. And I’m not opposed to the closed individualist view: I’m working on problems in value alignment for AI, I’m signed up for cryonics, and I think better decision theory is worth having. After all, I think closed individualism is right, and not just partially right, but all right, up to the limit of not being willing to say it’s right to the exclusion of the other perspectives. I think the closed individualism view feels real to people and is accurately describing both people’s experiences of individuality and some of the phenomena that create it.
So why am I sad? In many ways, closed individualism is a view that is built on suffering. It contains within it a great loneliness for creatures like us who want desperately to connect. It says that no matter how hard we try to bridge the gap it will always remain, and many people feel that if they were given the chance to eliminate it and merge with others they wouldn’t want to because then they’d lose themselves. To be a closed individualist is to live in fear: fear of death, fear of change, fear of loss of control. To me, that’s sad, because there’s another way.
The closed individualist might object “so what, this is a teleological argument: I might not want it to be that I am isolated and suffer, but closed individualism is what the world looks like, and I can’t be hurt by what is already true, so I maintain this position is right”. But I think this is wrong, because closed individualism is “wrong” in the sense that it doesn’t tell the whole story. If you’re looking for the theory that’s the most scientifically defensible, that for sure is empty individualism, not closed individualism, but it’s also very hard to get an intuitive grasp on empty individualism that you can live with and not sometimes think of yourself as a closed individual, so this tends to leave even the person who believes empty individualism is right acting as if closed individualism is how the world works.
The way out lies through open individualism, but this is a hard one to write about. Until you’ve felt the joy of open hearted connectedness to all being with ever fiber of existence, I think you’d have a hard time taking this view seriously, and the only way to feel this and take it seriously is probably through hundreds if not thousands of hours of meditation (you can also feel it with drugs but I think it’s more likely a person would dismiss or misunderstand the feeling as just a cool thing they felt on drugs). The “I am not it but it is me” sense you get is not really possible to explain to others; you have to see it for yourself because it exists somewhere beyond distinction such that it can never be brought back to a world carved up into more than one whole.
So here we are, trapped in a world of suffering, persisted because every closed individualist suffers and generates more suffering for the whole because it is in all of us. Thus am I sad.
[upvoted for talking about something that’s difficult to model and communicate about]
Hmm. I believe (with fairly high confidence—it would take a big surprise to shift me) a combination of empty and closed. Moments of self-observed experience are standalone, and woven into a fabric of memories in a closed, un-sharable system that will (sooner than I prefer) physically degrade into non-experiencing components.
I haven’t found anyone who claims to be open AND is rational enough to convince me they’re not just misstating what they actually experience. In fact, I’d love to hear someone talk about what it means to “want” something if you’re experiencing all things simultaneously.
I’m quite sympathetic to the argument that it is what it is, and there’s no reason to be sad. But I’m also unsure whether or why my acceptance of closed-empty existence makes you sad. Presumably, if your consciousness includes me, you know I’m not particularly sad overall (I certainly experience pain and frustration, but also joy and optimistic anticipation, in a balance that seems acceptable).
But I’m also unsure whether or why my acceptance of closed-empty existence makes you sad.
Because I know the joy of grokking the openness of the “individual” and see the closed approach creating inherent suffering (via wanting for the individual) that cannot be accepted because it seems to be part of the world.
I wonder how much the “great loneliness for creatures like us” is a necessary outcome of realizing that you are an individual, and how much it is a consequence of e.g. not having the kinds of friends you want to have, i.e. something that you wouldn’t feel under the right circumstances.
From my perspective, what I miss is people similar to me, living close to me. I can find like-minded people, but they live in different countries (I met them on LW meetups). Thus, I feel more lonely than I would feel if I lived in a different city. Similarly, being extraverted and/or having greater social skills could possibly help me find similar people in my proximity, maybe. Also, sometimes I meet people who seem like they could be what I miss in my life, but they are not interested in being friends with me. Again, this is probably a numbers game; if I could meet ten or hundred times more people of that type, some of them could be interested in me.
(In other words, I wonder whether this is not yet another case of “my personal problems, interpreted as a universal experience of the humankind”.)
Yet another possible factor is the feeling of safety. The less safe I feel, the greater the desire of having allies, preferably perfect allies, preferably loyal clones of myself.
Plus the fear of death. If, in some sense, there are copies of me out there, then, in some sense, I am immortal. If I am unique, then at my death something unique (and valuable, at least to me) will disappear from this universe, forever.
My quick response is that all of these sources of loneliness can still be downstream of using closed individualism as an intuitive model. The more I am able to use the open model the more safe I feel in any situation and the more connected I feel to others no matter how similar or different they are to me. Put one way, every stranger is a cousin I haven’t met yet, but just knowing on a deep level that the world is full of cousins is reassuring.
Strong and Weak Ontology
Ontology is how we make sense of the world. We make judgements about our observations and slice up the world into buckets we can drop our observations into.
However I’ve been thinking lately that the way we normally model ontology is insufficient. We tend to talk as if ontology is all one thing, one map of the territory. Maybe these can be very complex, multi-manifold maps that permit shifting perspectives, but one map all the same.
We see some hints at the breaking of this ontology of ontology as a single map by noticing the way some people, myself included, have noticed you can hold multiple, contradictory ontologies and switch between them. And with further development there’s no switching, it just all is, only complex and with multiple projections that overlap.
But there’s more. What we’ve been talking about here has mostly been a “strong” form of ontology that seeks to say something about the being of the world, to reify it into type-objects that can be considered, but there’s also a “weak” kind of ontology from which ontology arises and which can exist without the “strong” version. It’s the ontology that I referenced at the start of the post, the ontology of discrimination and nothing else. So much of ontology is taking the discrimination and turning it into a full-fledged model or map, but there’s a weak notion of ontology that exists even if all we do is draw lines where we see borders.
I can’t recall seeing much on this in Western philosophy; I thought about this after combining my reading of Western philosophy with my reading of Buddhist philosophy and what it has to say about how mental activity arises. But Buddhist philosophy doesn’t have a strong notion of ontology the way Western philosophy does, so maybe it’s not surprising this subtle point has gone missed.
So long as shortform is salient for me, might as well do another one on a novel (in that I’ve not heard/seen anyone express it before) idea I have about perceptual control theory, minimization of prediction error/confusion, free energy, and Buddhism that I was recently reminded of.
There is a notion within Mahayana Buddhism of the three poisons: ignorance, attachment (or, I think we could better term this here, attraction, for reasons that will become clear), and aversion. This is part of one model of where suffering arises from. Others express these notions in other ways, but I want to focus on this way of talking about these root kleshas (defilements, afflictions, mind poisons) because I think it has a clear tie in with this other thing that excites me, the idea that the primary thing that neurons seek to do is minimize prediction error.
Ignorance, even among the three poisons, is generally considered more fundamental, in that ignorance appears first and it gives rise to attraction and aversion (in some models there is fundamental ignorance that gives rise to the three poisons, marking a separation between ignorance as mental activity and ignorance as a result of the physical embodiment of information transfer). This looks to me a lot like what perceptual control theory predicts if the thing being controlled for is minimization of prediction error: there is confusion about the state of the world, information comes in, and this sends a signal within the control system of neurons to either up or down regulate something. Essentially what the three poisons describe is what you would expect the world to look like if the mind were powered by control systems trying to minimize confusion/ignorance, nudging the system toward and away from a set point where prediction error is minimized via negative feedback (and a small bonus, this might help explain why the brain doesn’t tend to get into long-lasting positive feedback loops: it’s not constructed for it and before long you trigger something else to down-regulate because you violate its predictions).
It also makes a lot of sense that these would be the root poisons. I think we can forgive 1st millennium Buddhists for not discovering PCT or minimization of prediction error directly, but we should not be surprised that they identified the mental actions this theory predicts should be foundational to the mind and also recognized that they were foundational actions to all others. Elsewhere, Buddhism explicitly calls out ignorance as the fundamental force driving dukkha (suffering), though we probably shouldn’t assign too many points to (non-Madhyamaka) Buddhism for noticing this since other Buddhist theories don’t make this same claims about attachment and aversion and they are used concurrently in explication of the dharma.
Hogwarts Houses as Religions
Okay, this is just a fun nonsense idea I thought up. Please don’t read anything too much into it, I’m just riffing. Sorry if I’ve mischaracterized a religion or Hogwarts house!
What religion typifies each Hogwarts house?
I’ll start with Hufflepuff, which I think is aligned with Buddhism: treat everyone the same, and if you want salvation the only option is to do multiple lifetimes worth of work.
Next is Ravenclaw, which looks a lot like Judaism: there’s a system to the world, you gotta follow the rules, and also lets debate and research endlessly to understand the last corner of things we don’t yet understand.
For Gryffindor, I’m saying Christianity: some things are just right and true and worth going on crusades for, and if you want to get into heaven you just gotta believe hard enough (but also good works might help).
Okay, now for the tough one, Slytherin. I’m going to go with Sorting Hat Chats and say Slytherin is the house of in-group loyalty and base my decision purely on that rather than all the other cruft associated with this house. And if we’re going by that, then I think we gotta link up Slytherin with Islam: if you’re in you’re family; if you’re not in but in-adjacent you’ll get second-class treatment that’s still pretty fair; if you’re out then you’re out and all bets are off.
Now obviously there’s lots of caveats here, like none of these “religions” I’ve picked out are really one single thing but rather multiple traditions and practices some of which don’t fit the pattern above well. Look, I get it. I’m just trying to point at something like the core thing associated with each of these religions that may be deviated from by various branches, e.g. Quakers are obviously Hufflepuffs, atheist are I guess Ravenclaws, LDS is probably Slytherin based on my reasoning above, etc.. This is just meant to be fun and sorry if I offended anyone who comes across this.
If CAIS if sufficient for AGI, then likely humans are CAIS-style general intelligences.
What’s the justification for this? Seems pretty symmetric to “If wheels are sufficient for getting around, then its’ likely humans evolved to use wheels.”
Human brains look like they are made up of many parts with various levels and means of integration. So if it turns out to be the case that we could build something like AGI via CAIS, that is CAIS can be assembled in a way that result in general intelligence, then I think it’s likely that human intelligence doesn’t have anything special going on that would meaningfully differentiate it from the general notion of CAIS other than being implemented in meat.
Won’t I get bored living forever?
I feel like this question comes up often as a kind of push back against the idea of living an unbounded number of years, or even just a really really long time beyond the scale of human comprehension for what it would mean to live that many years.
I think most responses rely on intuition about our lives. If your life today seems full of similar days and you think you’d get bored, not living forever or at least taking long naps between periods of living seems appealing. Alternatively, if your life today seems full of new experiences, you’d expect to keep having new experiences if you lived a long time. That’s probably a good description of why most people believe what they do.
But I think we can make a better argument for why living a long time wouldn’t result in your getting bored.
Much of the idea that you’d get bored is predicated on the idea that one day (so much as that remains a coherent concept in the future) is much like another. In fact, maybe you live the same day many times and that seems boring and not worth living. Yet days can only be the same from one’s understanding of them, or else be literally the same day and thus there would be no reason not to keep reliving it.
To explain, fundamentally no moment in the universe seems to be the same as any other moment because universe moments vary on dimensions that guarantee this. Yes, there may be many quite similar moments, but now two moments are literally the same. The only exception to this is that you might run a simulation in the universe that creates days that are identical from within the frame of the simulation. But if you do this then there’s no problem with living the same day multiple times, because if the same day is literally the same day that means you must be the same on each of these days and would thus have no realization you were reliving the same day (i.e. it’s “Groundhog day” but no one realizes it’s a loop). Yes, to us this might look like a kind of wireheading and might prefer that not be how we live, but from the inside we wouldn’t object and if we did it wouldn’t matter because the day would just reset and our objection we arise afresh each day without anything ever changing or us ever getting tired of living the same day over and over.
Thus we are left with days that are in fact unique, even if we believe them to be the same, and so we can get bored only to the extent we don’t care about the uniqueness of each day. This seems quite satisfying to me as a reason to want to live forever, to get to see the unfolding of events over billions of years or longer, but even if it’s not I think it should give you hope that there might be something to do over billions of years since notions of sameness of days are likely either limitations of human imagination or the result of being trapped in a looping simulation (which, again, you shouldn’t care about from the inside of the simulation).
When people barely live 100 years and we worry about them getting bored if they could live forever… that seems to me like finding a beggar who only has $100 net worth and is asking for some spare change, and explaining to him that giving him more money would be bad because eventually he would become a billionaire and everyone knows that power corrupts. Yeah, it has some philosophical merit, but is completely unrelated to the life as we know it.
For me, this looks like a very simplified treatment (I mean in a I-need-to-simplify-to-model-it way; I wanted to avoid the word ‘academic’). While the word boredom as you seem to use it is a very practical and complex emotion. I can’t disagree with your model but I don’t think it captures what people feel is boring now or what would be boring in the future. I think a good counterpoint is the one by Yoav, that you can just go to sleep until something new comes up. Something that is not possible if your time is limited, to begin with.
When this argument is presented to me, there are two counterpoints I often use:
Simple induction. I wasn’t bored enough to want to die yesterday, nor the day after that (today). Assuming that future days are roughly as similar as the past two, that degree of novelty is sufficient.
Options are not commitments. If I ever do want to die, I can do so. If it never happens, or doesn’t happen for a thousand or a hundred thousand years, that’s fine too.
For those who really want to engage on #2, I’ve had interesting conversations about akrasia-like self-disagreements where “I am bored and would prefer to have died” but “I have FOMO and will not willingly die”. For this, there is a possibility of mechanism design, where the decision can be made rule-based. Something like “after N years (say, 3⁄4 the median lifespan of your reference group), take a permanent poison, such that you must take an antidote every week/year. If you ever get bored/unhappy enough to not take the antidote, you die.)
A tougher disagreement is the Malthusian one—old people are already too powerful, and it’ll get far worse if they’re healthy and active for centuries (let alone longer). Further, they take resources/opportunities from the young. The availability heuristic for this is vampires, not techno-utopia. I have yet to really find a good counterargument for this—it quite likely contains a fair grain of truth, at least for the current planetary and human governance limitations.
Another option on two is to go into some kind of preservation instead of total death forever. then you can write instructions on when to wake you up (if X person asks, if X event happens, in X years, etc..). you still miss out on some stuff, but not literally everything.
The second benefit is that for people who stayed it’s not like you died and they’ll never be able to interact with you again, you just took a really long vacation :)
If powerful old people go to sleep when they are bored then they run the risk of being overtaken by younger, faster, and less risk-averse people. Maybe a good model is corporations: Corporations are also immortal and can learn more and more but they also have more to lose and they seem to acquire knowledge that also seems to slow them down. If there are changes in the environment or innovations they often cannot adapt fast enough and are quickly overtaken by younger players.
Personality quizzes are fake frameworks that help us understand ourselves.
What-character-from-show-X-are-you quizzes, astrology, and personality categorization instruments (think Big-5, Myers-Briggs, Magic the Gathering colors, etc.) are perennially popular. I think a good question is to ask, why do humans seem to like this stuff so much that even fairly skeptical folks tend to object not to categorization but that the categorization of any particular system is bad?
My stab at an answer: humans are really confused about themselves, and are interested in things that seem to have even a little explanatory power to help them become less confused about who they are. Metaphorically, this is like if we lived in a world without proper mirrors, and people got really excited about anything moderately reflective because it let them see themselves, if only a little.
On this view, these kinds of things, while perhaps not very scientific, are useful to folks because they help them understand themselves. This is not to say we can totally rehabilitate all such systems, since often they perform their categorization by mechanisms with very weak causal links that may not even rise above the level of noise (*cough* astrology *cough*), nor that we should be satisfied with personality assessments that involve lots of conflation and don’t resolve much confusion, but on the whole we should be happy that these things exist because they help us see our psyches in the absence of proper mental mirrors.
(FWIW, I do think there is a way to polish you mind into a mirror that can see itself and that I have managed to do this to some extent, but that’s a bit besides the point I want to make here.)
They help us understand others as well—even as fake frameworks, anything that fights against https://wiki.lesswrong.com/wiki/Typical_mind_fallacy is useful. I’d argue these categorizations don’t go far enough, and imply a smaller space of variation than is necessary for actual modeling of self or others, but a lot of casual observers benefit from just acknowledging that there IS variation.
As I work towards becoming less confused about what we mean when we talk about values, I find that it feels a lot like I’m working on a jigsaw puzzle where I don’t know what the picture is. Also all the pieces have been scattered around the room and I have to find the pieces first, digging between couch cushions and looking under the rug and behind the bookcase, let alone figure out how they fit together or what they fit together to describe.
Yes, we have some pieces already and others think they know (infer, guess) what the picture is from those (it’s a bear! it’s a cat! it’s a woman in a fur coat!), and as I work I find it helpful to keep updating my own guess because even when it’s wrong it sometimes helps me think of new ways to try combining the pieces or to know what pieces might be missing that I should go look for, but it also often feels like I’m failing all the time because I’m updating rapidly based on new information and that keeps changing my best guess.
I suspect this is a common experience for folks working on problems in AI safety and many other complex problems, so I figured I’d share this metaphor I recently hit on for making sense of what it is like to do this kind of work.
Most of my most useful insights come not from realizing something new and knowing more, but from realizing something ignored and being certain of less.
After seeing another LW user (sorry, forgot who) mention this post in their commenting guidelines, I’ve decided to change my own commenting guidelines to the following, matching pretty close to the SSC commenting guidelines that I forgot existed until just a couple days ago:
Comments should be at least two of true, useful, and kind, i.e. you believe what you say, you think the world would be worse without this comment, and you think the comment will be positively received.
I like this because it’s simple and it says what rather than how. My old guidelines were all about how:
Seek to foster greater mutual understanding and prefer good faith to bad, nurture to combat, collaboration to argument, and dialectic to debate. Do that by:
-aiming to understand the author and their intent, not what you want them to have said or fear that they said
-being charitable about potential misunderstandings, assuming each person is trying their best to be clearly understood and to advance understanding
-resolving disagreement by finding the crux or synthesizing contrary views to sublimate the disagreement
I’m fairly tolerant, but if you’re making comments that are actively counterproductive to fruitful conversation by failing to read and think about what someone else is saying I’m likely to ask you to stop and, if you don’t, delete your comments and, if you continue, ban you from commenting on my posts.Some behavior that is especially likely to receive warnings, deletions, and bans:
-trying to “score points”
-hitting “applause lights”
-being contrarian for its own sake
More generally, I think the SSC commenting guidelines might be a good cluster for those of us who want LW comment sections to be “nice” and so mark our posts as norm enforcing. If this catches on this might help deal with finding the few clusters of commenting norms that make people want without having lots of variation between authors.
I similarly suspect automation is not really happening in a dramatically different way thus far. Maybe that will change in the future (I think it will), but it’s not here yet.
So why so much concern about automation?
I suspect because of something they don’t look at in this study much (based on the summary): displacement. People are likely being displaced from jobs into other jobs by automation or the perception of automation and some few of those exit the labor market rather than switch into new jobs. Further, those who do move to new jobs likely disprefer their new jobs because they require different skills, they are less skilled at them immediately after switching, and due to lack of initial skill these new jobs initially pay less than the old jobs. This creates compelling evidence for the automation “destroying” jobs story even though the bigger picture makes it clear that this isn’t really happening, in particular because the destroying job story ignores the contrary evidence from what happens after a worker has been in a new job after displacement for a few years and have recovered to pre-displacement levels of wages.
This post suggests a feature idea for LessWrong to me:
It would be pretty cool if, instead of a lot of comments that have an order determined by votes or time of posting it were instead possible to write a post that had part that could be commented on directly. So, for example, say the comments for a particular section could live straight in the section rather than down at the bottom. Could be an interesting way to deal with lots of comments on large, structured posts.
You have reinvented Google Docs.
A similar effect could be achieved by having a sequence which...all appears on one page. (With the comments.)
Medium also has this feature, and I think it improves the Medium discourse quite a bit.
Commeting on specific parts of articles and seeing those comments as you go through the article.
I few months ago I found a copy of Staying OK, the sequel to I’m OK—You’re OK (the book that probably did the most to popularize transactional analysis), on the street near my home in Berkeley. Since I had previously read Games People Play and had not thought about transactional analysis much since, I scooped it up. I’ve just gotten around to reading it.
My recollection of Games People Play is that it’s the better book (based on what I’ve read of Staying OK so far). Also, transactional analysis is kind of in the water in ways that are hard to notice so you are probably already kind of familiar with some of the ideas in it, but probably not explicitly in a way you could use to build new models (for example, as far as I can tell notions of strokes and life scripts were popularized by if not fully originated within transactional analysis). So if you aren’t familiar with transactional analysis I recommend learning a bit about it since although it’s a bit dated and we arguably have better models now, it’s still pretty useful to read about to help notice patterns of ways people interact with others and themselves, sort of like the way the most interesting thing about Metaphors We Live By is just pointing out the metaphors and recognizing their presence in speech rather than whether the general theory is maximally good or not.
One things that struck me as I’m reading Staying OK is its discussion of the trackback technique. I can’t find anything detailed online about it beyond a very brief summary. It’s essentially a multi-step process for dealing with conflicts in internal dialogue, “conflict” here being a technical term referring to crossed communication in the transactional analysis model of the psyche. Or at least that’s how it’s presented. Looking at it a little closer and reading through examples in the book that are not available online, it’s really just poorly explained memory reconsolidation. To the extent it’s working as a method in transactional analysis therapy, it seems to be working because it’s tapping into the same mechanisms as Unlocking the Emotional Brain.
I think this is interesting both because it shows how we’ve made progress and because it shows that transactional analysis (along with a lot of other things), were also getting at stuff that works, but less effectively because they had weaker evidence to build on that was more confounded with other possible mechanisms. To me this counts as evidence that building theory based on phenomenological evidence can work and is better than nothing, but will be supplanted by work that manages to tie in “objective” evidence.
Off-topic riff on “Humans are Embedded Agents Too”
One class of insights that come with Buddhist practice might be summarized as “determinism”, as in, the universe does what it is going to do no matter what the illusory self predicts. Related to this is the larger Buddhist notion of “dependent origination”, that everything (in the Hubble volume you find yourself in) is causally linked. This deep deterministic interdependence of the world is hard to appreciate from our subjective experience, because the creation of ontology creates a gulf that cuts us off from direct interaction, causing us to confuse map and territory. Much of the path of practice is learning to unlearn this useful confusion that allows us to do much by focusing on the map so we can make better predictions about the territory.
In AI alignment many difficulties and confusions arise from failing to understand what there is termed embeddedness, the insight that everything happens in the world not alongside it on the other side of a Cartesian veil. The trouble is that dualism is pernicious and intuitive to humans, even as we deny it, and unlearning it is not as simple as reasoning that the problem exists. Our thinking is so polluted with dualistic notions that we struggle to see the world any other way. I suspect if we are to succeed at building safe AI, we’ll have to get a lot better at understanding and integrating the insight of embeddedness.
Psychological Development and Age
One of the annoying things about developmental psychology is disentangling age-related from development-related effects.
For example, as people age they tend to get more settled or to more have their lives sorted out. I’m pointing at the thing where kids and teenages and adults in their 20s tend to have a lot of uncertainty about what they are going to do with their lives and that slowly decreases over time.
A simple explanation is that it’s age related, or maybe more properly experience related. As a person lives more years, tries more things, and better learns the limits of what they can achieve they become more settled into the corner of the world in which they can successfully operate. We should expect people with less years of experience to be more confused about themselves and older people to be less confused because they had more years to figure things out. This need not invoke developmental psychology, just simple accumulation of evidence.
But counter examples of resigned young people and curious and adventurous old people abound. But perhaps we can explain these simply in terms of contexts that cause that, and if a person were put in a different context they might behave differently and in a more age anticipated way.
Where I think developmental psychology helps is in understanding subtler differences in how people behave and why. For example, is a person content with their life because they’ve tried a bunch of things and given up stretching too far because they know it’ll just be hard to do new things and they’ll probably fail so it’s not worth the effort, or are they content with their life because they trust themselves. It can be kinda hard to tell from the outside and you might not even be able to tell with an intervention, say by making something easier for them so they can do some new thing that they previously could not do easily and seeing how they respond.
This is one of the challenges I think we face in talking about developmental psychology. It seems a useful model for explaining many aspects of how minds progress over time, but it’s hard to figure out what’s conflated with things that would happen in a world with no developmental psychology beyond simple learning.
(Don’t worry; I haven’t suddenly found developmental psychology less useful, just musing a bit on an issue that comes up often of the form “this developmental model sounds fine as far as it goes, but what about this simpler explanation”.)
I’m not sure I fully endorse this idea, hence short form, but it’s rattling around inside my head and maybe we can talk about it?
I feel like there’s a kind of ADHD (or ADD) expansionism happening, where people are identifying all kinds of things as symptoms of ADHD, especially subclinical ADHD.
On the one had this seems good in the sense that performing this kind of expansionism seems to actually be helping people by giving them permission to be the way they are via a diagnosis and giving them strategies they can try to live their life better.
On the other I feel like it’s terrible in terms of actually diagnosing ADHD. It might help to explain why I think that.
Much of what I see that I’m terming ADHD expansionism looks to me like taking normal human behavior that is ill fitted to the modern environment and then pathologizing it. As best I can tell, it’s normal and adaptive for humans to exhibit various behaviors that that get labeled as ADHD symptoms, like flittering between multiple activities, hyperfocus on things the mind finds important but doesn’t necessarily endorse as important (S1 important things, not S2 important), understimulation, overstimulation, and otherwise finding it hard to focus on a one thing.
All of that sounds like normal, adaptive, forager behavior to me. Some of it became maladaptive during the farming era, but not especially, and now in the industrial era are less adaptive.
Thus I think ADHD suffers from the same issue as codependency does, in that if you start to describe the symptoms you quickly realize 90% of humanity has this “problem” and so I think we’re doing ourselves a disservice by considering it a pathology because it fails to acknowledge that most of these mental habits are just what it’s like to be a normal human and that its our conditions that are unusual and that we are struggling to function within.
I don’t see this as cause to throw out modern industrial society, but rather that we need to think about ways to adapt our systems to better accommodate real humans rather than the idealized ones of high modernism.
On the ground level, yes, we may still need to do much to personally intervene against ADHD-like symptoms, just as we may need to do against our natural tendency towards codependency, but I think there’s something being lost by even talking about it this way. Rather, we need to think of it as how do we cope with being humans engaged in systems that ask us to behave in unusual ways, and see the systems as the broken things, not ourselves. It’s not that everyone has ADHD or codependency; rather, it’s that our systems pathologize normal behavior because they are confused about what is typical.
There’s not much agreement on what to call it when “normal” is harmful, but not so overwhelmingly common as to seem immutable. Agreed that thinking of it as a pathology doesn’t quite cut it, but also “acceptable” seems wrong.
You’re always doing your best
I like to say “you’re always doing your best”, especially as kind words to folks when they are feeling regret.
What do I mean by that, though? Certainly you can look back at what you did in any given situation and imagine having done something that would have had a better outcome.
What I mean is that, given the all conditions under which you take any action, you always did the best you could. After all, if you could have done something better given all the conditions you would have.
The key is that all the conditions include the entire history of the world up to the present moment, and so that necessarily includes your life history, the life history of others, the physical environment, your emotional state, how tired you were, how your brain works, etc.. The trick is that when you condition your actions so fully there’s no room left for any counterfactuals, for you could have done nothing else!
As you might guess, I’m proposing a deterministic outlook on the world. I won’t really argue that too much here, other than to say that if you look long and hard enough at free will it dissolves into an after-the-fact illusion contingent on how your brain compresses reality and models yourself and that this is robust to quantum effects since even if quantum effects result in random outcomes you nonetheless only ever find yourself in a single history where some particular thing happened regardless of how it happened.
The immediate corollary of all this is that you also are always doing your worst, only that doesn’t land too well when someone feels regret.
I like this insight because, fully taken it, it dissolves regret. Not that you can’t imagine having done better, propose things you might do differently in the future, and then try them to see what happens and maybe actually do better than you previously did. Rather, it dissolves regret because regret hinges on feeling as if a counterfactually could have really happened. Once you deeply believe that counterfactuals are not real, i.e. they are purely of the map and have no existence in the territory independent of the map, regret just has no way to come into existence.
This doesn’t mean you can’t still feel related emotions like remorse, especially if you realize you were negligent and had a responsibility to have done better but didn’t, but that’s different than clinging to a desire to have done something different; remorse is owning that you did something less than what you were capable of under the circumstances and might reasonably be asked to make amends.
So next time you feel regret, try reminding yourself it couldn’t have gone any other way.
I feel like something is screwy with the kerning on LW over the past few weeks. Like I keep seeing sentences that look like they are missing space between the period and the start of the next sentence but when I check closely they are not. For whatever reason this doesn’t seem to show in the editor, only in the displayed text.
I think I’ve only noticed this with comments and short form, but maybe it’s happening other places? Anyway, wanted to see if others are experiencing this and raise a flag for the LW team that a change they made may be behaving in unexpected ways.
It is totally real and it’s been this way over two months. It’s an issue with Chrome, and I’m kinda boggled that Chrome doesn’t jump on these issues, it’s a big deal for readability.
Yep, it’s a Chrome bug. It’s kind of crazy.
Story stats are my favorite feature of Medium. Let me tell you why.
I write primarily to impact others. Although I sometimes choose to do very little work to make myself understandable to anyone who is more than a few inferential steps behind me and then write out on a far frontier of thought, nonetheless my purpose remains sharing my ideas with others. If it weren’t for that, I wouldn’t bother to write much at all, and certainly not in the same way as I do when writing for others. Thus I care instrumentally a lot about being able to assess if I am having the desired impact so that I can improve in ways that might help serve my purposes.
LessWrong provides some good, high detail clues about impact: votes and comments. Comments on LW are great, and definitely better in quality and depth of engagement than what I find other places. Votes are also relatively useful here, caveat the weaknesses of LW voting I’ve talked about before. If I post something on LW and it gets lots of votes (up or down) or lots of comments, relative to what other posts receive, then I’m confident people have read what I wrote and I impacted them in some way, whether or not it was in the way I had hoped.
That’s basically where story stats stop on LessWrong. Here’s a screen shot of the info I get from Medium:
For each story you can see a few things here: views, reads, read ratio, and fans, which is basically likes. I also get an email every week telling me about the largest updates to my story stats, like how many additional views, reads, and fans a story had in the last week.
If I click the little “Details” link under a story name I get more stats: average read time, referral sources, internal vs. external views (external views are views on RSS, etc.), and even a list of “interests” associated with readers who read my story.All of this is great. Each week I get a little positive reward letting me know what I did that worked, what didn’t, and most importantly to me, how much people are engaging with things I wrote.
I get some of that here on LessWrong, but not all of it. Although I’ve bootstrapped myself now to a point where I’ll keep writing even absent these motivational queues, I still find this info useful for understanding what things I wrote that people liked best or found most useful and what they found least useful. Some of that is mirrored here by things like votes, but it doesn’t capture all of it.
Very quick thought: basically the reasons we haven’t and might not do more in this direction is how it might alter what gets written. It doesn’t seem good if people were to start writing more heavily for engagement metrics. Also not clear to me that engagement metrics capture the true value that matters of intellectual contributions.
(Habryka has an old comment somewhere delving into this, which I couldn’t find. But the basic gist was “the entire rest of the internet is optimizing directly for eyeballs, and it seemed good for LessWrong to be a place trying to have a different set of incentives”)