Can you please update on footnotes? I notice people have written posts with footnotes, so it must be possible, but I can’t figure out how. Thanks in advance
steve2152
Karma: 38 (LW), 0 (AF)
1hr talk: Intro to AGI safety
I haven’t seen anyone argue for CRT the way you describe it. I always thought the argument was that we are concerned about “rational AIs” (I would say more specifically, “AIs that run searches through a space of possible actions, in pursuit of a real-world goal”), because (1) We humans have real-world goals (“cure Alzheimer’s” etc.) and the best way to accomplish a real-world goal is generally to build an agent optimizing for that goal (well, that’s true right up until the agent becomes too powerful to control, and then it becomes catastrophically false), (2) We can try to build AIs that are not in this category, but screw up*, (3) Even if we here all agree to not build this type of agent, it’s hard to coordinate everyone on earth to never do it forever. (See also: Rohin’s two posts on goal-directedness.)
In particular, when Eliezer argued a couple years ago that we should be mainly thinking about AGIs that have real-world-anchored utility functions (e.g. here or here) I’ve always fleshed out that argument as: ”...This type of AGI is the most effective and powerful type of AGI, and we should assume that society will keep making our AIs more and more effective and powerful until we reach that category.”
*(Remember, any AI is running searches through some space in pursuit of something, otherwise you would never call it “intelligence”. So one can imagine that the intelligent search may accidentally get aimed at the wrong target.)
This paper replaces a normal feedforward image classifier with a mesa-optimizing one (build generative models of different possibilities and pick the one that best matches the data). The result was better and far more human-like than a traditional image classifier, e.g. the same examples are ambiguous to the model that are ambiguous to humans and vice-versa. I also understand that the human brain is very big into generative modeling of everything. So I expect that ML systems of the future will approach 100% mesa-optimizers, while non-optimizing feedforward NN’s will become rare. This post is a good framework and I’m looking forward to follow-ups!
My view is that the “constrict blood vessels and tense muscles” action (or whatever it is) is less like moving your finger, and more like speeding up your heart rate: sorta consciously controllable but not by a simple and direct act of willful control. I personally was talking to my hands rather than talking to my subconscious, but whatever, either way, I see it as a handy trick to send out the right nerve signals. Again like how if you want to release adrenaline, you think of something scary, you don’t think “Adrenal gland, Activate!” (Unless you’ve specifically practiced.)
I guess where I differ in emphasis from you is that I like to talk about how an important part of the action is really happening at the location of the pain, even if the cause is in the brain. I find that people talking about “psychosomatic” tend to be cutting physiology out of the loop altogether, though you didn’t quite say that yourself. The other different emphasis is whether there’s any sense whatsoever in which some part of the person wants the pain to happen because of some ulterior motive. I mean, that kind of story very much did not resonate with my experience. My RSI flare-ups were always pretty closely associated with using my hands. I guess I shouldn’t over-generalize from my own experience. Shrug.
“Constricting blood vessels” seems like a broad enough mechanism to be potentially applicable to back spasms, RSI, IBS, ulcers, and all the other superficially different indications we’ve all heard of. But I don’t know much about physiology or vasculature, and I don’t put too much stock in that exact description. Could also be something about nerves I guess?
Ten years ago I had a whole miserable year of Repetitive Strain Injury (and other things too) and then had a miraculous one-day recovery after reading Sarno’s book. But I didn’t (and still don’t) agree with the way Sarno (and you) describe what the mechanism is and what the emotions are doing, at least for my own experience and the couple other people I know personally who had similar RSI experiences.
I think it’s possible to use muscles in a way that’s painful: something like having the muscles be generally tense and their blood supply constricted. I think state-of-mind can cause muscles to operate in this painful blood-supply-constricted mode, and that one such state of mind is the feeling “These muscles here are super-injured and when I use them it’s probably making things worse.”
If you think about it, it’s easy to think of lots of things could predispose people to get stuck in this particular vicious cycle, including personality type, general stress level, factual beliefs about the causes and consequences of chronic pain, and so on. But at the end, I think it’s this pretty specific thing, not a catch-all generic mechanism of subconscious expression or whatever like you seem to be thinking.
Hi John, Are you saying that there should be more small teams in AGI safety rather than increasing the size of the “big” teams like OpenAI safety group and MIRI? Or are you saying that AGI safety doesn’t need more people period?
Looks like MIRI is primarily 12 people. Does that count as “large”? My impression is that they’re not all working together on exactly the same narrow project. So do they count as 1 “team” or more than one?
The FLI AI grants go to a diverse set of little university research groups. Is that the kind of thing you’re advocating here?
ETA: The link you posted says that a sufficiently small team is: “I’d suspect this to be less than 15 people, but would not be very surprised if this was number was around 100 after all.” If we believe that (and I wouldn’t know either way), then there are no AGI safety teams on Earth that are “too large” right now.