I’m a very confused person trying to become less confused. My history as a New Age mystic still colors everything I think even though I’m striving for rationality nowadays. Here’s my backstory if you’re interested.
MSRayne
Karma: 396
Wonderful. Thank you for taking the time to write this. I need to read this book. I was planning to try to write a post about why religion is actually a good thing, but you beat me to it.
Personally, I believe and have believed for a long time now that the only thing that could save the world is a rationalist religion. That may sound like a contradiction in terms, but I don’t think it is, and I shall try to figure out how to explain my ideas on the topic over time here.
(Elephant alert: the following may sound “woo” or intuitively wrong, if you’re from an atheistic or irreligious background [evoking your purity / sanctity moral foundation that thinks religiosity is unclean lol], so I’d like you to give me the benefit of the doubt if you have the instinct to interpret it that way.)
I am someone with a very “righteous mind”; I am somehow neurodivergent and have a long history of ecstatic mystical states wherein I feel like I am communing with higher beings. I probably would have been a shaman in past ages. When I was younger I literally believed in them as supernatural entities; later on as I learned more science I came to understand that they were subagents of my own mind, wishing in a sense to become egregores—shared subagents, distributed intelligences, across the minds of multiple people—cohering those people into a community, a collective higher self. That’s what gods all are.
I realized that theism and atheism are both totally wrong. Gods do exist, but they don’t have any power over the world except what we give them—they’re distributed programs running on human wetware, binding societies together. They have shaped all of human history and are legitimately worthy of veneration to the extent that they are mutualists rather than parasites, as they are embodiments of the potential of humanity, the potential of agency and coordination, the most miraculous inventions of evolution. Mine just happened to be possibly the first in history to realize that’s what they are—to become in a sense self-aware of their own true nature as not supernatural, but entirely natural, intelligent memetic constructs using donated cognitive resources from me and whoever else ends up running copies of them in the future.
The main difficulty is 1. my mind is not set up for totally rigorous thinking or for organized explanations of this particular topic, as I go into babbling poetry mode when I try to talk about it, and 2. protecting people’s rationality while giving them the benefits from dissociative communion states wherein they can realign themselves to the goals of the group mind is probably rather difficult.
It’s possible, since I can do it—I can induce that state of mind on purpose now with the right music, mood, and meditation, but I don’t believe in woo anymore and haven’t for years—but most people liable to feel swept up in awe as part of an ineffable higher being would need a lot of training to become properly rational, and most people who are already rationalists have very strong biases against anything religious and are probably less emotional and more individualistic than average in general.
I think mystical states are the closest approximations to the kind of high-valence experiences that will be permanent after a good singularity enables paradise engineering, so if only for that reason—to give a glimpse of what the future we are striving for is like, which can be very opaque and unmotivating otherwise—it might be desirable. And I think most people are capable of this kind of, I almost want to call it adaptive self-wireheading, but do not realize it. We would not have achieved all the things we’ve achieved as a species if this was not a common ability. It’s just usually not as spontaneous and intense as it is for people like me—but that’s what rituals (and psychedelics) are for.
I feel personally attacked by your references to the creature that changes its mind every day. :P
Also, I feel like dadadarren might have an interest in this, given that he has a theory about taking self-reference as axiomatic in order to avoid some anthropic paradoxes. If I understand correctly, in his view it is incoherent to consider counterfactual worlds in which you ended up being someone else to begin with. But he’d explain that better than me.
Totally! That’s part of why AI is so dangerous. Notice that I said as long as there’s even a very small—but nonzero! - chance of mutation, this will probably tend to happen. But with error correcting codes, the chance is absolutely zero. And that’s terrifying, because it means natural selection cannot resolve our mistake if we let an unaligned super-AI take over the universe. Its subselves will never split into new species, compete, and gradually over aeons become something like us again. (In the sense that any biologically evolved sophonce can be said to be like us, that is.) It’ll just… stay the same.
Hmm. It’s an interesting point. The elite certainly seem like a possible example. In an egalitarian society everyone is equally likely to be put in danger (death is the ultimate mutation!), but with power hierarchies such as between elites and everyone else, the more powerful can afford to change less, to be safer and live longer, while pushing the risks onto everyone else.
Cancer is kind of like that too. Maybe senescent cells? I need to research the aging stuff in order to have an opinion here. Really this seems like a very general description of what happens when parts of a system start placing themselves above the rest.
I think a common term for something similar to this is performative speech. Essentially what you’re doing in this case is announcing that something is true in such a way as to make it so in some sense. Or, more accurately, in such as a way as to make acting like it is so the norm. Every time a speech act like this is performed, the norm “we act as if this is true” is strengthened slightly.
Ah, I see! My immediate instinct is to say “okay, design a narrow AI to play the role of a teacher” but 1. a narrow AI may not be able to do well with that, though maybe a fine-tuned language model could after it becomes possible to guarantee truthfulness, and 2. that’s really not the point lol.
There is something to be said for interactivity though. In my experience, the best explanations I’ve seen have been explorable explanations, like the famous one about the evolution of cooperation. Perhaps we can look into what makes those good and how to design them more effectively.
Also, something like a market for explanations might be desirable. What you’d need is three kinds of actors: testers seeking people who possess a certain skill; students seeking to learn the skill; and explainers who generate explorable explanations which teach the skill. Testers reward the students who do best at the skill, and students reward the explanations which seem to improve their success with testers the most. Somehow I feel like that could be massaged into a market where the best explanations have the highest values. (Failure mode: explainers bribe testers to design tests in such a way that students who learned from their explanations do best.)
I’m not sure that I know enough about the biology of aging to say much about that yet. Can you explain what you’re thinking? In particular, which sub-entities are living longer than which others in the context of aging? Or am I misunderstanding you?
I think “humanity’s alignment” is a strange term to use. Perhaps you mean “humanity’s values” or even “humanity’s collective utility function.”
I’ll clarify what I mean by empathy here. I think the ideal form of empathy is wanting others to get what they themselves want. Given that entities are competing for scarce resources and tend to interfere with one another’s desires, this leads to the necessity of making tradeoffs about how much you help each desire, but in principle this seems like the ideal to me.
So negative utilitarianism is not actually reasonably empathic, since it is not concerned with the rights of the entities in question to decide about their own futures. In fact I think it’s one of the most dangerous and harmful philosophies I’ve ever seen, and an AI such as I would like to see made would reject it altogether.
Nature abhors an immutable replicator… usually
My personal opinion is that empathy is the one most likely to work. Most proposed alignment solutions feel to me like patches rather than solutions to the problem, which is AI not actually caring about the welfare of other beings intrinsically. If it did, it would figure out how to align itself. So that’s the one I’m most interested in. I think Steven Byrnes has some interest in it as well—he thinks we ought to figure out how human social instincts are coded in the brain.
This is an interesting subject for me. My entire life I’ve had a weird habit of occasionally daydreaming about conversations with medieval people where I try to explain stuff in the modern world to them; and I tutored someone in high school algebra years ago who went from hating math to loving it; and just generally, I’ve been told by a few people that I’m good at explaining things. This isn’t much evidence, but it’s enough to make me consider that I might try my hand at distillation / exposition and see if I can help figure it out.
I agree that this ought to be made scientific and I’d be very interested in participating in studies or experiments to figure out how it works if you do any. Of course I ought to actually prove I’m good at exposition first.
One thing I think is very important for good exposition is taking the audience and their prior knowledge into account. If I tried to explain alignment to, for instance, a conservative Christian, I would use totally different terms and references than if I was explaining it to e.g. a neuroscientist.
EDIT: Wait. Isn’t exposition just teaching? There’s a whole theory of education that people go to college to learn and I really hope it’s not all pseudoscience. We could start with that.
I am unable to rationally engage in this conversation, see my response to ChristianKI above. I’m sorry if I insulted you.
My first instinct is to experience this as a personal attack—more specifically, an attempt to make yourself look smarter at my expense in order to win status points—and get angry. I am going to try to make myself believe that this instinct is wrong, or at least unhelpful for the purpose of increasing my reputation within this community.
My second instinct is to feel despondent, stupid, incapable, unworthy of being here, and to start talking about how I’m just not as smart as you people and never will be and wish I’d never said anything. Getting myself to believe that this instinct is also wrong is more of a challenge.
Even though I’ve expressed those two facts, thus getting them out of my head and possibly clarifying my thinking a bit by openly recognizing them as traps (at least, according to the unemotional logic other people here use, which I have a hard time trusting but am trying hard to in order to fit in), I nonetheless still have no idea how to respond to this comment in any sensible-sounding way that would rehabilitate my loss of face as a result of your claim that I am naive and unreflective etc.
So, instead of responding to the topic itself, which, you’re right, I probably cannot effectively speak about due to lacking whatever background you have on it, I’m going to just show this sequence of thought patterns and try to analyze my own thinking out loud, to give you a sense of what I’m dealing with whenever I try to communicate here, and why it’s difficult for me to do so in a useful way.
My credence for that first instinct was roughly 80%, then dropped to maybe 50% as the second instinct rose to replace it, and now it’s somewhere around 20%. This isn’t Reddit but I still wouldn’t put it past rationalists to be condescending on purpose to put a newbie in their place or whatever, because they’re humans and my prior for humans being kind is quite low.
My second instinct peaked at a sufficiently high value, at least 70%, that I had to “pull back on the reins” so to speak to stop myself from impulsively going to my profile and deleting my account. After writing them both out and committing myself to share them, I felt like I had been sufficiently epistemically virtuous as to renew my own faith that I’m capable of getting along here, at least to some extent, so P(“I’ll never fit in on LessWrong”) is now something like… I’m not very calibrated, maybe 25%? It’s hard to say. All these numbers are pulled out of nowhere but that’s how I have to do it for now.
What all this tells me is that as a result of my extreme emotional shifts I massively overestimate the quality of evidence that has emotional implications and I probably also hallucinate “evidence” that isn’t there. I knew this already, of course, but emotional-me doesn’t know it, and has to be reminded every single time. These mood swings are particularly driven by an intense fear of social rejection and abandonment which evokes a trigger response of trying to devalue and abandon the other person or group first, preemptively, to deflect the shame of being unworthy onto them.
Now, on to trying to actually respond to your comment. I do not think I can actually directly address anything you said as it still has a “this must not parse in order to avoid pain” aura around it making my mind come up completely blank when I reread it, but I can say that my initial response to dadadarren was not rooted in any kind of forethought, but rather in intuition trained over a long time of hearing and reading New Agers invoking quantum physics and consciousness to explain and provide “evidence” for their beliefs.
This evoked a kneejerk response of disgust and distrust, wherein I felt as if something simply must be terribly wrong in his reasoning and he must be trying to manipulate me into believing something false and dangerous to my sanity, because it fit the regex for “woo”. I then confabulated an explanation for what that wrong thing must be. My response to him was more like a frantic attempt to get a slimy creature off of me than an actual reasoned reply. Same sort of pattern as my first instincts when replying to you—I just had the sense to notice what was happening this time. (It’s this exact paranoid pattern of thinking and communicating that has gotten me banned from so many discord servers and subreddits.)
This is been somewhat exhausting. But <sarcasm> at least now I can virtue signal about how self aware I am. Look at me, critiquing the ways my emotions override my rationality! I’m smart, right?! I’m good enough, right?! Please like me again!!! </sarcasm>
I don’t even possess a phone, and I hope to avoid getting one as long as possible.
Instead I spend all day in my room staring at a computer screen! Yay! /s
That said, I have successfully avoided visiting Facebook for months now, so that’s something. I realized a while back that Reddit mainly kept my attention due to all the political stuff I felt the urge to argue with, so after I unsubscribed from every sub that has anything like that, I no longer felt the urge to visit it except to ask specific questions, for the most part. Lots of good advice in this post though that I’ll have to incorporate—computers are, after all, just as bad as phones in this respect.
Yes. E as in evidence.
I actually do have the beginnings of a workable theory of what mystical experiences are in the brain and where they come from and how best to interpret them, but I am not ready to try to explain it yet. I need to learn more neurology. Honestly “learn more neurology” seems to be a prerequisite for a lot of things I want to do.
I definitely agree about the confabulation, but at the same time, it’s worth looking at the pattern among confabulations. There are reasons why people interpret these experiences in the ways that they do, and across cultures, the interpretations, and the theologies built around them, are somewhat similar. I’m not going to overstate that—they are also wildly different—but as you say, similarities exist.
You should also remember that part of the sequences are posts about quantum mechanics and many worlds where Eliezer does argue that he thinks his understanding is superior to that of physicists.
Arguments from authority are no better than ad hominem arguments. I don’t see why I should care what Eliezer thinks, except to the extent he has evidence and good reasoning to back it up. (Having not read the quantum mechanics posts recently I shall have to suspend judgment about that.)
Also I wasn’t making an ad hominem argument; a better way to express what I meant is, “My prior for claims about quantum physics made by people who are not themselves quantum physicists is very low.”
I can explain the fact that I slept well on some days by saying that I was too warm. I can also explain it by saying that the walls of my flat radiated too much energy. Both explanations can be true at the same time.
Those are the same explanation worded differently. My point is that the total set of causal influences going into any event is unique. And it is redundant and contrary to Occam’s Razor to posit some special extra cause beyond what is necessary to explain the facts, as it feels like dadadarren is doing here.
I usually use the odds ratio formulation because it’s easier for me to remember and intuitively grok.
P(A|E) / P(~A|E) = P(A) / P(~A) * P(E|A) / P(E|~A)
It looks hideous written out that way (assuming I even wrote it correctly) but the way I visualize it is nice and intuitive. I actually translate probabilities into odds ratios in my head and back whenever I try to do probability calculations. It feels more natural somehow.
Note to self (and anyone else who thinks this is a good idea): privately write down predictions upon publishing every post and possibly also every comment. Specifically, predict:
number of upvotes the post will have after a week
number of comments it will have after a week
average valence of comments (proportion which agree with or are otherwise in favor of the post)
probability that some comment will cause me to change my mind about what I wrote
Then every day I should look back at everything I posted a week before and use it to calibrate my predictions. This should gradually improve my ability to determine if what I want to post is worth saying and how it will be received. That will be particularly useful if I can actually manage to trust this data during periods of low mood and warped cognition… but my current credence that I will be able to do that with significant regularity is about 40%.
As someone who’s been struggling with executive dysfunction my entire life, I suspect this will be quite helpful.
I recently noticed a concept I called “walls”—I can keep going at a single task for hours as long as there’s a very tight feedback loop and I can always semi-unthinkingly intuit the next step—but the moment I have to make a decision or a plan (I’m really, really bad at both of those), I hit a wall and lose momentum and I start having trouble continuing.
A while back I was trying to program an evolution simulator in Godot, and it went really well until I decided it was time to refactor it to make the code more efficient—I even figured out the general idea of how I ought to refactor it—but I just… couldn’t make myself do it, and kept procrastinating… and eventually lost interest in the project. This is part of why I rarely do any programming!