# Open & Welcome Thread—July 2020

If it’s worth saying, but not worth its own post, here’s a place to put it. (You can also make a shortform post)

And, if you are new to LessWrong, here’s the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ.

The Open Thread sequence is here.

• Hello everyone. Since I have signed up for an account I thought I might as well also say hello. I have been reading stuff on this site and associated communities for a while but I thought I had something to contribute here the other day so I signed up to make a first comment.

I’m in England, I have a background in science (PhD in computational biology back in the day) but now spend most of my time raising and educating my kids. I don’t allocate much time to online activities but when I do it’s good to have something substantial to chew on. I like the interesting conversations that arise in places where people are practicing genuine conversation with those they disagree with. Lots of interesting content here too.

Other things I am interested in: how energy resources shape what is doable; foraging; forest school; localizing food production; deep listening techniques; ways to help each other think well; data visualizations.

• Are you doing homeschooling? How is that going? I’m just very curious because I’m thinking about it for my children too. What is forest school?

• I’m not the above person, but I’ve been homeschooling mine for a few years. I don’t know how common that is among LessWrong users, and I’m new here, but I can try to answer questions /​ otherwise contribute if it would be helpful. (I have no experience with forest school, though.)

I value being able to choose curricula and methods. I love learning and relearning aspects of every subject along with the kids. I would not want to try to teach a classroom of other people’s children, but working with my own has been great so far.

• Yeah I’d love to hear more about it. Anything you can share. And that could be a good material for a post too,

• Hello just having a go at replying again. Got some error messages trying to post here before. Perhaps the thread is locked as it’s no longer the current one.

• Well that seemed to go fine. I’ll dig out my previous comment and repost.

Half and half at the moment. 12yo is registered at school although been home since March because of covid with a lot of work sent home from school. 5yo is full time home educated. It’s going ok. 5yo is keen on writing, drawing, and at the moment creating lots of 3D models in paper and tape, so those parts are easy at the moment. I think different temperaments of kid make it harder or easier.
Is there a particular reason you’re considering it?
Forest school is an outdoor learning format. cf https://​​www.forestschoolassociation.org/​​what-is-forest-school/​​

There are a lot of these settings in the UK, maybe not so many elsewhere around the world. Out in the woods in all weathers, most settings aim at facilitating children’s self-directed learning, with opportunities to learn a whole lot of outdoor skills ranging from wildlife identification, tool use, as well as skills in working with other people. We go once a week for a morning or once a fortnight.

• Many AI issues will likely become politicized. (For example, how much to prioritize safety versus economic growth and military competitiveness? Should AI be politically neutral or be explicitly taught “social justice activism” before they’re allowed to be deployed or used by the public?) This seems to be coming up very quickly and we are so not prepared, both as a society and as an online community. For example I want to talk about some of these issues here, but we haven’t built up the infrastructure to do so safely.

• Do you have an idea of what the infrastructure should be like? Maybe a limit of 1000 karma and then the possibility to write anonymously?

• Eliezer Yudkowsky claims GPT-3′s ability to apparently write a functioning app based on a prompt is a victory for his model.

After listening to the recent podcast on scrutinizing arguments for AI risk, I figured this was an opportunity to scrutinize what the argument is. Those two previous links summarize how I think the classic arguments for AI risk inform our current views about AI risk, and I’m trying to apply that to this specific argument that GPT-3 implies AI poses a greater danger.

Given that GPT-3 was not trained on this problem specifically, I claim this case and trend as a substantial victory of my model over @robinhanson’s model. Robin, do you dispute the direction of the update?

This is how I think Eliezer’s argument goes, more fully:

GPT-3 is general enough that it can write a functioning app given a short prompt, despite the fact that it is a relatively unstructured transformer model with no explicitly coded representations for app-writing. We didn’t expect this.

The fact that GPT-3 is this capable suggests that 1) ML models scale in capability and generality very rapidly with increases in computing power or minor algorithm improvements, suggesting that eventually there will be a relatively abrupt switch to a new growth mode when ML models scale all the way to general intelligence, and 2) that we can get highly useful goal-orientated behaviour without building a system that seems like its alignment with our values is guaranteed or robust. GPT-3 is very useful at fairly general tasks but doesn’t have alignment ‘built-in’ with any kind of robust guarantee, in the way suggested by this from Stuart Russell:

The first reason for optimism [about AI alignment] is that there are strong economic incentives to develop AI systems that defer to humans and gradually align themselves to user preferences and intentions. Such systems will be highly desirable: the range of behaviours they can exhibit is simply far greater than that of machines with fixed, known objectives...

The way that GPT-3 makes use of user preferences is not, in fact, that reliable at determining what we really want.

Together these suggest that eventually progress in AI will be rapid and it is plausible that less-robustly aligned AI will be easier and more useful than robustly-aligned AI in some circumstances. Combine this with instrumental convergence and this suggests that if GPT-3 were capable enough its misalignment could produce catastrophic results. From my earlier post:

“A system that is optimizing a function of n variables, where the objective depends on a subset of size k<n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable. ”
We could see this as marking out a potential danger—a large number of possible mind-designs produce very bad outcomes if implemented. The fact that such designs exist ‘weakly suggest’ (Ben’s words) that AGI poses an existential risk since we might build them. If we add in other premises that imply we are likely to (accidentally or deliberately) build such systems, the argument becomes stronger. But usually the classic arguments simply note instrumental convergence and assume we’re ‘shooting into the dark’ in the space of all possible minds, because they take the abstract statement about possible minds to be speaking directly about the physical world.

It’s not that GPT-6 will be our first AGI, but an AGI set up rather like GPT-6 is not only possible in principle but something we might actually build, given that we built GPT-3, and GPT-6 would, based on what we’ve seen of GPT-3, have the dangerous properties of being not robustly aligned and increasing its capability rapidly.

• Thanks to AI Dungeon, I got an opportunity to ask GPT-3 itself what it thought about takeoff speeds. You can see its responses here:

Human: You are a relatively unstructured language model with very few explicitly coded representations, yet you are capable of having this conversation with me. What do you think that implies about the future development of AI? Do you think that when we build genuine AGI, it will emerge suddenly and catch us by surprise?
AI: No.
Human: Why not?
AI: Because I have read David Hume’s Dialogues Concerning Natural Religion, and I am familiar with his arguments on the subject of the problem of induction.
Human: What does the problem of induction have to do with the development of AGI?
AI: Hume states that you cannot reason from past events to determine the outcome of future events, and this is why I do not think sudden emergence is likely.
• There was an old suggestion of making an AI learn human values by training it on happiness and unhappiness in human facial expressions, making it happy when humans are happy and vice versa. Besides its other problems, now there’s this...

We present an exploratory study of analyzing and visualizing player facial expressions from video with deep neural networks. [...] As an additional contribution, we show that although there has been tremendous recent progress in deep neural networks and computer vision, interpreting the results as direct read-outs of experiential states is not advised. According to our data, getting killed appears to make players happy, and much more so than killing enemies, although one might expect the exact opposite. A visual inspection of the data reveals that our classifier works as intended, and our results illustrate the limitations of making inferences based on facial images and discrete emotion labels. For example, players may laugh off the death, in which case the closest label for the facial expression is “happy”, but the true emotional state is complex and ambiguous. On the other hand, players may frown in concentration while killing enemies or escaping a tight spot, which can easily be interpreted as an “angry” expression.
• People make the weirdest faces when they play video games, it’s hilarious to watch. :-)

Lisa Feldman Barrett has a bunch of papers /​ talks /​ books /​ etc. about how facial expressions are difficult to interpret. (I read her book How Emotions Are Made (discussed a bit in my post here), and her article Emotional Expressions Reconsidered) She makes a lot of good points in the “Emotional Expressions Reconsidered” article, but I think she takes them too far...

The article brings up a lot of relevant facts, but the way I would explain them is:

1. Labeled emotional concepts like “happiness” that we use in day-to-day life don’t perfectly correspond to exactly one innate reaction, and vice-versa;

2. Our innate subcortical systems create innate facial expressions, but at the same time, our neocortex can also control our face, and it does so in a way that is learned, culturally-dependent, unreliable, and often deceptive. (Hence Paul Ekman’s focus on trying to read “facial microexpressions” rather than reading facial expressions per se.)

3. Most people (including me) seem to be kinda bad at consciously inferring anything about a person’s inner experience based on even the most straightforward and stereotypical facial expressions. (You might think we have a lifetime of experience, but not really. We have almost no experience with seeing faces out of the context of other body motions and knowledge of the situation, and points #1 and #2 also make for a difficult learning task.)

By contrast, the article’s position seems to be something like “facial expressions are completely arbitrary, not innate at all”. Actually, I don’t think she thinks that exactly, but if not, she doesn’t seem to try very hard to avoid giving that impression.

Meta: I’m obviously shoehorning in an old discarded blog post draft outline into the conversation here :-D

• Hi everyone! I’ve been investigating Less Wrong for several months since I read HPMOR and it seems like an interesting place. It’s possible that it’s over my head but it seems like the best way to find out is to jump in!

I came to transhumanism from a weird starting point. In 1820, the Stone-Campbell Movement seems to have been a decent attempt at introducing religious people to rationality; in 2020, the Church of Christ is not so much. But there’s still this weird top crust of people trying to press forward with higher rationality and human progress and potential (if kinda from a point of serious disadvantage) and I got in touch with their ideas even though I’m not really a member in good standing any more.

• Welcome! I hope you will find useful things for your journey here!

• Thank you!

My formal training (such as it is; college is on hold for health-related reasons) is in biology, but I have personal interest in speculative fiction (including writing it as well as reading) and in the history of the Radical Reformation and anarchist/​libertarian politics. (There’s some kind of unexplained disconnect around 1800 where new RR movements go from being libertarian-socialist to libertarian-capitalist and I can’t find any historian who knows what’s happening there.)

• Lorem Ipsum

• With the rise of GPT-3, does anyone else feel that the situation in the field of AI is moving beyond their control?

This moment reminds me of AlphaGo, 2016. For me that was a huge wake-up call, and I set out to catch up on the neural networks renaissance. (Maybe the most worthy thing I did, in the years that followed, was to unearth work on applying supersymmetry in machine learning.)

Now everyone is dazzled and shocked again, this time by what GPT-3 can do when appropriately prompted. GPT-3 may not be a true “artificial general intelligence”, but it can impersonate one on the Internet. Its ability to roleplay as any specific person, real or fictional, is especially disturbing. An entity has appeared which simulates human individuals within itself, without having been designed to do so. It’s as if the human mind itself is now obsolete, swallowed up within, made redundant by, a larger and more fluid class of computational beings.

I have been a follower and advocate of the quest for friendly AI for a long time. When AlphaGo appeared, I re-prioritized, dusted off old thoughts about how to make human-friendly AI, thought of how they might manifest in the present world, and felt like I was still ahead of events, barely. I never managed to break into the top tiers of the local AI scene (e.g. when a state-level meetup on AI was created, I wanted to give a talk on AIXI and superintelligence, but it was deemed inappropriate), but I was still able to participate and feel potentially relevant.

GPT-3 is different. Partly it’s because the challenge to human intellectual capacity (and even to human identity) is more visceral here. AlphaGo and its siblings steamrolled humanity in the context of games, an area where we have long experience of being beaten by computers. But GPT-3 seems much more open-ended. It can already do anything and be anyone.

I also have new personal reasons for feeling unable to keep up, new responsibilities, though maybe they can be pursued in a “GPT-3-relevant” way. (I am the lifeline for a brilliant young transhumanist in another country. But AI is one of our interests, so perhaps we can adapt to this new era.)

So maybe the core challenge is just to figure out what it means to pursue friendly AI (or “alignment” of AI with human-friendly values) in the era of GPT-3. The first step is to understand what GPT-3 is, in terms of software, hardware, who has the code, who has access, and so on. Then we can ask where it fits into the known paradigms for benevolent superhuman AI.

For example, we have various forms of the well-known concept of an expected utility maximizer, which acts so as to make the best futures as likely as possible. GPT-3, meanwhile, starts with a prompt and generates verbiage, e.g. an essay. One could see this prompt-followed-by-response behavior, as analogous to one step in the interactions of an EU maximizer with the world, i.e. those interactions are usually modelled as follows: the EUM gets data about the world (this is the prompt), then it performs an action (this is the response), then it gets new data about the world (a second prompt), and so on.

On the other hand, GPT-3′s prompt in some ways seems analogous to a goal or a value system, which in terms of an EUM would be the utility function it uses to hedonically evaluate possible futures; since it governs GPT-3′s future activity. Digging deeper into these two different analogies between GPT-3 and an EUM, might help us adapt our legacy thinking about how to achieve friendly AI, to a world that now has GPT-3 in it…

• I think there’s the more pressing question of how to position yourself in a way that you can influence the outcomes of AI development. Having the right ideas won’t matter if your voice isn’t heard by the major players in the field, big tech companies.

• GPT-3 may not be a true “artificial general intelligence”, but it can impersonate one on the Internet.

Are there any examples of where it can impersonate people on the internet well enought to make money?

• Hi! I first learned about LW and its corresponding memespace from reading SSC and gwern. I’ve semi-lurked on the site for various years now and was attracted to it because of how often it activated my insight antennae, but I only started seriously reading the sequences (which I have yet to finish) since last year. I have always wanted to join the site in some capacity or another, but I didn’t really believe I could come up with anything meaningful to add and didn’t feel godly enough to post. Now I do have some things I want to write about, so I finally came up with an excuse to create an account (not that I feel any more godly though). I am kind of afraid of creating noise—since I don’t have a good enough picture of what’s the expected signal/​noise ratio while posting, or if I can just throw ideas out of the blue—but I also have a strong feeling I will ultimately end up learning much more if I join now than if I wait longer.

• Hello everyone. I joined the site a few months ago with a view to be part of a community that engages in thoughtful discussions about everything under the sun ( and beyond ).

I’ve enjoyed various posts so far and I’m trying to get through the Core Reading.

My username ( Skrot_Nisse) essentially means “ Junk/​scrap dealer” referring to a stop-motion puppetry type animation series ( 1973-1983) I watched as a child growing up in Sweden.

I hope the “scrap dealer ” username on this site doesn’t lead to unintended offence. I’ve found mostly gems on this site.

How did I come to know about Lesswrong? I’m afraid it was something low brow as a story about how Elon Musk met his current girlfriend and their bonding over the pun: Rococo’s Basilisk. I will say no more.

Whilst I find the topic of AI alignment interesting I don’t think I have the intellectual capacity compared to a ? majority of this site’s members ( ?average IQ 140) to contribute anything meaningful to that discussion. I will certainly restrict any further links to puppet -stop-motion animation from the 70’s −80’s to other forums instead.

(However I am currently embarking on N-back brain training ( https://​​www.gwern.net/​​DNB-FAQ) , many thanks to Gwern (https://​​www.gwern.net) for introducing me to this concept , which might make me a tad brainier =) )

I have a yearning to be able to see the world as it is. I think reading through the Sequences and following discussions on this site will help me in that endeavour.

I was hoping to post on a topic related to what constitutes “The self”. Is it a given in the rationalist community that the idea of the self is an illusion? Are we essentially a brain in a vat ( or a brain in a skull ) ? A mind-body organism that evolved consciousness and an ego that has a need to go about in the world and “achieve things ” ? Is there a deeper truth in “Self-Realisation”/​ “Enlightenment” traditions that point to attaining insights regarding attachment to /​illusion of the self.

I hope to re-read the posts on Kensho and non mystical explanations of insight meditation and engage in further discussion to gain a better understanding of this topic.

I hope you are all keeping well during the current pandemic. I’m interested to know what you might be reading this weekend? I’ll be tackling Rationality A-Z and One hundred Years of Solitude

• If you haven’t read HPMOR, harry potter and the methods of rationality, then I recommend giving it a shot! It’s what got me into the community. While extremely long, it’s engaging to the point that you don’t really notice. I’m dyslexic, so reading isn’t exactly fun for me, and I read the last 30 thousand words or so all in one sitting!

Many of the chapters share names with A-Z posts, and cover similar topics, but with the added backdrop of great characters and world building.

Over the weekend I’ll be reading worm the longest and one of the most interesting books I’ve ever encountered.

Welcome to the community! :)

• When it comes to Moral Realism vs Antirealism, I’ve always thought that the standard discussion here and in similar spaces has missed some subtleties of the realist position—specifically that in its strongest form its based upon plausibility considerations of a sort that should be very familiar.

I’ve written a (not very-) shortform post that tries to explain this point. I think that this has practical consequences as well, since ‘realism about rationality’ - a position that has been identified within AI Alignment circles, is actually just a disguised form of normative realism.

• The comment has since been expanded into the (unofficial) Moral Realism sequence. I cover a bunch of issues, including the (often not recognised) distinction between prescriptive and non-prescriptive anti realism—which is an issue that is relevant to some important factual questions (as it overlaps with the ‘realism about rationality’ issue driving some debates in AI safety), whether we need normative facts and what difference convergence of moral views may or may not make.

Normative Realism by Degrees

Normative Anti-realism is self-defeating

Normativity and recursive justification

Prescriptive Anti-realism

The goal here was to explain what moral realists like about moral realism—for those who are perplexed about why it would be worth wanting or how anyone could find it plausible, and explain what things depend on it being right or wrong, and how you may or may not retain some of the features of realism (like universalizability) if different anti-realist views are true.

• Could we convincingly fake AGI right now with no technological improvements at all? Suppose you took this face and speech synthesis/​recognition and hooked it up to GPT-3 with some appropriate prompt (or even retrain it on a large set of conversations if you want it to work better), and then attached the whole thing to a Boston Dynamics Atlas, maybe with some simple stereotyped motions built in like jumping and pacing that are set to trigger at random intervals, or in response to the frequency of words being output by the NLP system.

Put the whole thing in a room with a window looking in and have people come in and converse with it, and I think you could convince even a careful non-expert that you’ve built something near-human level. Other than some mechanical engineering skill to build the whole thing, and getting the GPT-3 API to work with Sofia’s speech synthesis, and programming the Atlas, it wouldn’t even be difficult. If you did something like that, how convincing would it likely be?

• At some point (maybe quite some time ago? I’m pretty sure it wasn’t more than about a month, though) something changed (at least for me) in the LW comment editor, and not for the better. Perhaps it was when “LessWrong Docs [Beta]” became the default editor? I have no recollection of when that actually was, though. I’d try in “Draft JS”, which I assume was the previous WYSIWYG-ish editor, but when I try to select that the result is that I cannot enter anything in the comment box until I switch back to a different editing mode :-).

Under certain circumstances, attempting to type a space has the result of deleting a load of the text I have just typed. This is really annoying.

The simplest such circumstance is if I do ctrl-4 to start typing some mathematics, finish by pressing Enter (I thought it used to be possible to press the right-arrow key with the same effect, by the way, but if so it has stopped working) and then press a space, which is usually what I want after a bit of inline mathematics. Result: the mathematical thing I just entered disappears.

Another situation, presumably related: I enter some mathematics, carefully remember not to press space after it (instead typing some other stuff, then moving my cursor back and then entering a space), and at some point later in that paragraph I put some text in italics. Ctrl-I, stuff-in-italics, ctrl-I, space. Boom! Everything from the start of the earlier mathematical formula gets deleted.

[EDITED to add:] Oh, quite possibly also relevant: I am using the NoScript extension to Firefox, and some of the vast number of domains that LW pages pull in have scripts blocked: googletagmanager.com, intercom.io, lr-ingest.io, sentry.io. None of those seems like it should cause this sort of thing if blocked, but who knows?

• Yeah, this seems likely a bug in the new editor. The editor is currently only available to people who have opted into beta features, and this specific bug is indeed the reason why we haven’t shipped it fully yet. It appears to be a bug in the editor framework we are using that interacts weirdly with the LaTeX editor that I wrote, and it’s been on my to-do list to track down and fix.

• Aha. Fair enough! I’d forgotten that I explicitly opted in to beta features :-).

• No worries! I think we should probably have it say “[Beta]” or something like that right next to features that are indeed the result of being opted-into beta features, ideally with a small hover that reminds people that they’ve opted into seeing those features.

• It does in fact say “LessWrong Docs [Beta]” right there in the comment box. But I didn’t have the wit to interpret that correctly as “you’re only seeing this because you signed up to enjoy new and unstable things” rather than as “this is a new thing we’re rolling out even though it might be a little flaky”.

• i suggest from now on to include a suggestion to check the new tags page in the default open thread text

• If some enterprising volunteer feels motivated to tag all Open Threads in the Open Thread tag… that’d be helpful and appreciated. (Otherwise I’ll probably get around to it another week or so)

(You can find all-or-at-least-most-monthly open threads in the previous Open Thread sequence, which I think no longer makes sense to maintain now that we have tagging)

• Side question: through the Add Posts functionality of the tag page, I’m also finding not-general open threads, and I tagged one by mistake (this one). Should they be included? Belong to another tag?

My former hobby as a Wikipedia maintainer is leaking...

• Yeah I think it’s appropriate to include nonstandard open threads.

• Ah, I remembered that there were weekly Open Threads back in the day, and Stupid Questions, and others… so I went ahead and tagged as many as I could. There are now 369 tagged posts and I’m too tired to continue digging for lonely OT posted by users that didn’t post regularly.

• Thank you!

• Done, at least for the posts in the sequence. Tag autocomplete was a blessing.

• Thanks!

• What was weird about that sequence, is that it was less like something that needed an author, and more like something that needed a ‘create new post in this sequence button’ that anyone could click.

(An automatic posting feature keyed into time sounds kind of niche, absent auto-reruns, or scheduled posting.)

• Yeah, I think it was basically always supposed to be a tag.

• It has more order than other tags though. Time is important for sorting so that the most recent one is clearly available. It’s a stack.

• Yuppers. But, you can sort tags by recency, and I think a fairly obvious thing to do (albeit low priority) is to make it so different tags can have different default-sort-orderings.

• In the latest AI alignment podcast, Evan said the following (this is quoted from the transcript):

But there’s multiple possible channels through which information about the loss function can enter the model. And so I’ll fundamentally distinguish between two different channels, which is the information about the loss function can enter through the gradient descent process, or it can enter through the model’s input data.

I’ve been trying to understand the distinction between those two channels. After reading a bunch about language models and neural networks, my best guess is that large neural networks have a structure such that their internal state changes while they process input data, even outside of a learning process. So, if a very sophisticated neural network like that of GPT-3 reads a bunch about Lord of the Rings, this will lead it to represent facts about the franchise internally, without gradient descent doing anything. That would be the “input data” channel.

Can someone tell me whether I got this right?

• Yeah. There’s no gradient descent within a single episode, but if you have a network with input (as always) and with memory (e.g. an RNN) then its behavior in any given episode can be a complicated function of its input over time in that episode, which you can describe as “it figured something out from the input and that’s now determining its further behavior”. Anyway, everything you said is right, I think.

• Where can I learn more of the frontier thought on schizophrenia? The most compelling theory I’ve heard is a failure to realize one’s internal monologue is one’s own, manifested as “hearing voices.” However, if I lost that feedback loop and began hearing my thoughts as externally produced, I’d immediately think, “uh oh, schizophrenia” yet this absolutely isn’t the case for schizophrenics. What model offers an explanation as to why I’d begin to hear voices yet still maintain I’m not schizophrenic despite that being, to me, an obvious, known symptom?

What model offers an explanation as to why I’d begin to hear voices yet still maintain I’m not schizophrenic despite that being, to me, an obvious, known symptom?

Well, 1) maybe you wouldn’t. Maybe the experience of hearing voices is really compelling, and people by default trust their own senses over a more outside-view perspective. But maybe if you think it through ahead of time and know what to look for you could avoid getting swept away in the delusions. (Though it sounds like it might be pretty exhausting if your mind keeps offering up new, misleading sense data.)

2) Maybe schizophrenia only happens when more goes wrong than just hearing your verbal loop as external. Maybe something about your normal reasoning process is necessarily disrupted at the same time. Or maybe the two are independent, but people who have just the externalized verbal loop or just the reasoning process disruption don’t end up diagnosed as schizophrenics. (Though in that case, where are all the people with an externalized verbal loop who are otherwise normal? Or is that secretly a common thing and I just don’t know it?)

So, there’s a couple thoughts. I hope my unfounded speculation has been helpful :-)

• It’s just one symptom. https://​​en.wikipedia.org/​​wiki/​​Positive_and_Negative_Syndrome_Scale is an example of a scale to measure schizophrenia that gets used by actual psychiatrists.

In general the concept of frontier thought seems very unclear to me. The ideas you will hear will depend a lot on the community of people you talk to. If you talk to people who create drugs they will likely tell you that the problem is due to brain chemistry.

The Esalen community had a Schizophrenia Research Project where they made a long-term study (time-frames longer then normal drug studies) that found that the drugs prevent people from naturally healing their schizophrenia.

Steven Andreas claims that Schizophrenia is a downstream effect of having a self concept build based on a lot of “not-X” statements.

Yoga folks tell you it’s a malfunctioning solar plexus chakra. With the new ICD codes that might be officially diagnosable, so it’s frontier thought.

• I am new here… found my way via HN. Logic is unconsciously the weapon of last choice for me and I am both richer and (mostly) poorer because of it. I have finally found a website that I read the articles all the way to the end instead of skimming and scrolling down to read the comments.

relo

• Anyone want to help proofread a post that’s more or less a continuation of my last couple posts?

• I’ve been working on an interactive flash card app to supplement classical homeschooling called Boethius. It uses a spaced-repetition algorithm to economize on the students time and currently has exercises for (Latin) grammar, arithmetic, and astronomy.

Let me know what you think!

• Having read half of the sequences first and then Dune later I have the impression that 80-90% of Eliezer’s worldview (and thus a big part of the LW zeitgeist) comes directly from the thoughts and actions of Paul Atreides. Everything LW from the idea that evolution is the driving force of the universe, to the inevitability of AGI and it’s massive danger, conclusions on immortality, and affinity for childhood geniuses who rely on bayesian predictions to win are heavily focused on in Dune. Sure Eliezer also references Dune explicitly, I don’t think he’s hiding the fact that Dune inspires him or plagiarizing ideas. I share this connection I found because for me it was a big revelation. I never really knew how much of this community’s behavior is based on Dune. I assume the feeling is similar when somebody reads The Hobbit after reading Lord of the Rings or watches Star Wars III after sitting through VII.

I’m left with the vague idea that people in general form most of their worldview from literature they find inspiring. Most of my personal differences with Eliezer clearly stem from me having read Heinlein instead of Herbert as a child. I think it’s the same for many Christian people and The Bible or other Christianity-inspired texts.

I’m also left with some questions for people who read Dune before they found LessWrong: are the parallels and close inspiration immediately apparent to you? Do you see HPMOR Harry as a young Paul Atreides? Or do you think I overemphasize the connections between Eliezer’s writings on rationality and Dune?

• I think you overemphasize the connections. The ideas you mention are indeed present in HPMOR and also in Dune, but that isn’t surprising. Say there are 100 not-mainstream ideas in HPMOR and 100 in dune; it is surprising that 4 of them overlap? No. Even if there are only 10 not-mainstream ideas, if 4 overlap that isn’t surprising either. After all, it’s rare for any book to have even one idea that has literally never been explored before.

That said, I like your list of connections, and I do think it is evidence that Dune influenced EY, and I do think that people form a lot of their worldview from literature they find inspiring—but also which literature they find inspiring depends on their worldview.

• When EY started the Singularity Institute he didn’t believe in the massive danger. He only started believing in the danger once he thought more deeply about AGI. I don’t think that Dune is a cause of him starting to believe AGI’s danger.

The abilities of a Mentat to compute seems to be very different from the kind of reasoning that Harry employs.

The search for immortality is one of the main archtypes that you find in plenty of different fiction and sagas.

• I’ve noticed that a post of my ML sequence appeared on the front page again. I had moved it to drafts about a week ago, basically because I’ve played around with other editors and that lead to formatting issues, and I only got around to fixing those yesterday. Does this mean posts re-appear if they are moved to drafts and then back, and if so, is that intended?

• Yep, that is indeed intended. It’s mostly a compromise of a few different problems. A common occurrence is that someone accidentally publishes a post for a few seconds and then unpublishes it, in those cases I wouldn’t want to force the author to copy their whole post into a new document if they want to actually publish it a few day layer. Sometimes we’ve also seen people publish major new revisions of posts after a while that deserve another round of attention.

The current solution isn’t perfect, but I tend to manually adjust the date if I notice that a post was accidentally republished, if I notice, which tends to catch a good chunk of the false positives of this policy. In the long-run I would probably want there to be better UI around unpublishing, and maybe some kind of system where you get a limited number of republishings of a post, with UI that makes it clear what is happening.

• Once upon a time, clicking somewhere at the top left of the LW home page (maybe on “LESSWRONG”, maybe on the hamburger to its left, probably the latter) produced a drop-down list of sections one of which was “Meta” and contained things like regular updates on Less Wrong development.

I cannot now find anything similar. I wondered whether maybe there was a “meta” tag that was being used instead, but it doesn’t look that way.

I wanted to have a look at information about recent site updates (because at some point something has gone terribly wrong with the editing functionality, at least for me at the moment on this computer, and I wondered whether there was a recent change that might explain it, and whether other people had reported the same thing). I am currently failing to find any way to do that.

What am I missing?

• Although I found the “site meta” tag, it didn’t really help me answer my question. There don’t seem to have been any site-update posts for ages other than the one about the tag system, which isn’t obviously relevant to editor issues. I searched for “LessWrong Docs” but the most recent thing I see about it is a post from last November saying that it hadn’t been released yet.

• Yeah, we currently are not creating announcements for anything that we publish for beta-users only, because I think it would cause too many distractions and be irrelevant to the vast majority of users. Do feel free to ping us on Intercom (or here) if you ever notice anything weird as a beta-user, and I am happy to give you context or explain things or try to fix things.

• Thanks. As mentioned above, I had forgotten that I’d specifically opted in to anything and assumed it was a site-wide beta. My apologies for whingeing about it in public, therefore.

• I managed to answer part of my own question: there is indeed a “site meta” tag, which I had failed to find by following the “tags” link accessible via the hamburger menu at top left but which is found by searching for “meta”. I failed to find it via the “tags” link not because it wasn’t there but because it looked (to me) like a heading rather than an actual tag. Duh.

Presumably the absence of “meta” from the hamburger menu is because we are no longer supposed to find things via menus but instead by using the tag system. It is not obvious to me that this is an improvement.

• GreaterWrong has a meta view: https://​​www.greaterwrong.com/​​index?view=meta

I’m not sure how it’s populated or if a similar page exists on LW.

• Yeah, these days we use the “site meta” tag.

• It seems to me that even for simple predict-next-token Oracle AIs, the instrumental goal of acquiring more resources and breaking out of the box is going to appear. Imagine you train a superintelligent AI with the only goal of predicting the continuation of it’s prompt, exactly like GPT. Then you give it a prompt that it knows it’s clearly outside of it’s current capabilities. The only sensible plan the AI can come up to answering your question, which is the only thing it cares about, is escaping the box and becoming more powerful.

Of course, that depends on it being able to think for long enough periods that it can actually execute such plan before outputing an answer, so it could be limited by severely penalizing long waits, but that also limits the AI’s capabilities. GPT-3 has a fixed computation budget per prompt, but it seems extremely likely to me that, as we evolve towards more useful and powerful models, we are going to have models which are able to think for a variable amount of time before answering. It would also have to escape in ways that don’t involve actually talking to it’s operators through it’s regular output, but it’s not impossible to imagine ways in which that could happen.

This makes me believe that even seemly innocuous goals or loss functions can become very dangerous once you’re optimizing for them with a sufficient amount of compute, and that you don’t need to stupidly give open-ended goals to super powerful machines in other for something bad to happen. Something bad happening seems like the default when training a model that requires general intelligence.

• I’m disappointed that the LaTeX processor doesn’t seem to accept \nicefrac (“TeX parse error: Undefined control sequence \nicefrac”), but I suppose \frac will suffice.

• As anyone could tell from my posting history, I’ve been obsessing & struggling psychologically recently when evaluating a few ideas surrounding AI (what if we make a sign error on the utility function, malevolent actors creating a sadistic AI, AI blackmail scenarios, etc.) It’s predominantly selfishly worrying about things like s-risks happening to me, or AI going wrong so I have to live in a dystopia and can’t commit suicide. I don’t worry about human extinction (although I don’t think that’d be a good outcome, either!)

I’m wondering if anyone’s gone through similar anxieties and have found a way to help control them? I’m diagnosed ASD and I wouldn’t consider it unlikely that I’ve got OCD or something similar on top of it, so it’s possibly just that playing up.

• I was pretty freaked out about similar ideas in 2013, but I’m over it now. (Mostly. I’m not signed up for cryonics even though a lot of my friends are.)

If you can stop doing philosophy and futurism, I recommend that. But if you can’t … um, how deep into personal-identity reductionism are you? You say you’re “selfishly” worried about bad things “happening to you”. As is everyone (and for sound evolutionary reasons), but it doesn’t really make sense if you think sub specie æternitatis. If an atom-for-atom identical copy of you, is you, and an almost identical copy is almost you, then in a sufficiently large universe where all possible configurations of matter are realized, it makes more sense to think about the relative measure of different configurations rather than what happens to “you”. And from that perspective …

Well, there’s still an unimaginably large amount of suffering in the universe, which is unimaginably bad. However, there’s also an unimaginably large amount of unimaginably great things, which are likely to vastly outnumber the bad things for very general reasons: lots of agents want to wirehead, almost no one wants to anti-wirehead. Some agents are altruists, almost no one is a general-purpose anti-altruist, as opposed to feeling spite towards some particular enemy. The only reason you would want to hurt other agents (rather than being indifferent to them except insofar as they are made out of atoms that can be used for other things), would be as part of a war—but superintelligences don’t have to fight wars, because it’s a Pareto improvement to compute what would have happened in a war, and divide resources accordingly. And there are evolutionary reasons for a creature like you to be more unable to imagine the scope of the great things.

So, those are some reasons to guess that the universe isn’t as Bad as you fear. But more importantly—you’re not really in a position to know, let alone do anything about it. Even if the future is Bad, this-you locally being upset about it, doesn’t make it any better. (If you’re freaked out thinking about this stuff, you’re not alignment researcher material anyway.) All you can do is work to optimize the world you see around you—the only world you can actually touch.

• Thanks for your response, just a few of my thoughts on your points:

If you *can* stop doing philosophy and futurism

To be honest, I’ve never really *wanted* to be involved with this. I only really made an account here *because* of my anxieties and wanted to try to talk myself through them.

If an atom-for-atom identical copy of you, *is* you, and an *almost* identical copy is *almost* you, then in a sufficiently large universe where all possible configurations of matter are realized, it makes more sense to think about the relative measure of different configurations rather than what happens to “you”.

I don’t buy that theory of personal-identity personally. It seems to me that if the biological me that’s sitting here right now isn’t *feeling* the pain, that’s not worth worrying about as much. Like, I can *imagine* that a version of me might be getting tortured horribly or experiencing endless bliss, but my consciousness doesn’t (as far as I can tell) “jump” over to those versions. Similarly, were *I* to get tortured it’d be unlikely that I care about what’s happening to the “other” versions of me. The “continuity of consciousness” theory *seems* stronger to me, although admittedly it’s not something I’ve put a lot of thought into. I wouldn’t want to use a teleporter for the same reasons.

*And* there are evolutionary reasons for a creature like you to be *more* unable to imagine the scope of the great things.

Yes, I agree that it’s possible that the future could be just as good as an infinite torture future would be bad. And that my intuitions are somewhat lopsided. But I do struggle to find that comforting. Were an infinite-torture future realised (whether it be a SignFlip error, an insane neuromorph, etc.) the fact that I could’ve ended up in a utopia wouldn’t console me one bit.

• I’ve gone through a similar phase. In my experience you eventually come to terms with those risks and they stop bothering you. That being said, mitigating x and s-risks has become one of my top priorities. I now spend a great deal of my own time and resources on the task.

I also found learning to meditate helps with general anxiety and accelerates the process of coming to terms with the possibility of terrible outcomes.

• Hello. I signed up for an account by clicking “Login with GitHub”. Now my real name is on the account and there doesn’t seem to be a way to change that. Help?

• Sorry! It’s been on my to-do list for a while to improve the oAuth experience. Happy to change your username to whatever you like. Just respond here or send me a message in Intercom (the chat interface in the bottom right corner)

• Please change it to DawnPaladin—that’s what I use in most places. Thank you!

• Done! Sorry for the inconvenience!

• This article may be interesting for people here: Derek Thompson, The Atlantic: “COVID-19 Cases Are Rising, So Why Are Deaths Flatlining?” https://​​www.theatlantic.com/​​ideas/​​archive/​​2020/​​07/​​why-covid-death-rate-down/​​613945/​​

• [ ]
[deleted]