Or you could think of misalignment as the AI doing things its designers explicitly tried to prevent it from doing (giving people suicide instructions and the like), then in this case the AI is clearly “misaligned”, and that says something about how difficult it’ll be to align our next AIs.
cousin_it(Vladimir Slepnev)
Can you describe what changed / what made you start feeling that the problem is solvable / what your new attack is, in short?
There’s an amazing HN comment that I mention everytime someone links to this essay. It says don’t do what the essay says, you’ll make yourself depressed. Instead do something a bit different, and maybe even opposite.
Let’s say for example you feel annoyed by the fat checkout lady. DFW advises you to step over your annoyance, imagine the checkout lady is caring for her sick husband, and so on. But that kind of approach to your own feelings will hurt you in the long run, and maybe even seriously hurt you. Instead, the right thing is to simply feel annoyed at the checkout lady. Let the feeling come and be heard. After it’s heard, it’ll be gone by itself soon enough.
Here’s the whole comment, to save people the click:
DFW is perfect towards the end, when he talks about acceptance and awareness— the thesis (“This is water”) is spot on. But the way he approaches it, as a question of choosing what to think, is fundamentally, tragically wrong.
To Mindfulness-Based Cognitive Therapy folks call that focusing on cognition rather than experience. It’s the classic fallacy of beginning meditators, who believe the secret lies in choosing what to think, or in fact choosing not to think at all. It makes rational sense as a way to approach suffering; “Thinking this way is causing me to suffer. I must change my thinking so that the suffering stops.”
In fact, the fundamental tenet of mindfulness is that this is impossible. Not even the most enlightened guru on this planet can not think of an elephant. You cannot choose what to think, cannot choose what to feel, cannot choose not to suffer.
Actually, that is not completely true. You can, through training over a period of time, teach yourself to feel nothing at all. We have a special word to describe these people: depressed.
The “trick” to both Buddhist mindfulness and MBCT, and the cure for depression if such a thing exists, lies in accepting that we are as powerless over our thoughts and emotions as we are over our circumstances. My mind, the “master” DFW talks about, is part of the water. If I am angry that an SUV cut me off, I must experience anger. If I’m disgusted by the fat woman in front of me in the supermarket, I must experience disgust. When I am joyful, I must experience joy, and when I suffer, I must experience suffering. There is no other option but death or madness— the quiet madness that pervades most peoples’ lives as they suffer day in and day out in their frantic quest to avoid suffering.
Experience. Awareness. Acceptance. Never thought— you can’t be mindful by thinking about mindfulness, it’s an oxymoron. You have to just feel it.
There’s something indescribably heartbreaking in hearing him come so close to finding the cure, to miss it only by a hair, knowing what happens next.
[Full disclosure: My mother is a psychiatrist who dabbles in MBCT. It cured her depression, and mine.]
And another comment from a different person making the same point:
Much of what DFW believed about the world, about himself, about the nature of reality, ran counter to his own mental wellbeing and ultimately his own survival. Of the psychotherapies with proven efficacy, all seek to inculcate a mode of thinking in stark contrast to Wallace’s.
In this piece and others, Wallace encourages a mindset that appears to me to actively induce alienation in the pursuit of deeper truth. I believe that to be deeply maladaptive. A large proportion of his words in this piece are spent describing that his instinctive reaction to the world around him is one of disgust and disdain.
Rather than seeking to transmute those feelings into more neutral or positive ones, he seeks to elevate himself above what he sees as his natural perspective. Rather than sit in his car and enjoy the coolness of his A/C or the feeling of the wheel against his skin or the patterns the sunlight makes on his dash, he abstracts, he retreats into his mind and an imagined world of possibilities. He describes engaging with other people, but it’s inside his head, it’s intellectualised and profoundly distant. Rather than seeing the person in the SUV in front as merely another human and seeking to accept them unconditionally, he seeks a fictionalised narrative that renders them palatable to him.
He may have had some sort of underlying chemical or structural problem that caused his depression, but we have no real evidence for that, we have no real evidence that such things exist. What we do know is that patterns of cognition that he advocated run contrary to the basic tenets of the treatment for depression with the best evidence base—CBT and it’s variants.
OpenAI has already been the biggest contributor to accelerating the AI race; investing in chips is just another step in the same direction. I’m not sure why people keep assuming Altman is optimizing for safety. Sure, he has talked about safety, but it’s very common for people to give lip service to something while doing the opposite thing. I’m not surprised by it and nobody should be surprised by it. Can we just accept already that OpenAI is going full speed in a bad direction, and start thinking what we can/should do about it?
Contrarians of LW, if you want to be successful, please don’t follow this strategy. Chances are that many people have raised the same possibility before, and anyway raising possibilities isn’t Bayesian evidence, so you’ll just get ignored. Instead, try to prove that the stuff is bullshit. This way, if you’re right, others will learn something, and if you’re wrong, you will have learned something.
My #1 suggestion, by a big margin, is to generate more new formal math results.
My #2 suggestion is to communicate more carefully, like Holden Karnofsky or Carl Shulman. Eliezer’s tone is sometimes too preachy.
I really liked your report of the scientology class. The conclusions, not so much. Many LW posts (including some of mine, too ashamed to link here) follow this pattern of giving a wonderful convincing anecdote and then a big flimsy over-generalization on top. Perhaps we could institute a norm that posting anecdotes without making conclusions from them is okay. I took some boxing lessons, still cannot fight but no longer fear physical confrontations, and that’s all. I learned to draw using a book by Betty Edwards, it was easy and fun, and that’s all.
- 22 Apr 2011 16:23 UTC; 4 points) 's comment on Reading Nonfiction Selectively by (
I don’t want LW to change in that direction.
In the famous talk “You and Your Research”, Richard Hamming explained why physicists don’t spend much time on researching antigravity:
The three outstanding problems in physics, in a certain sense, were never worked on while I was at Bell Labs. By important I mean guaranteed a Nobel Prize and any sum of money you want to mention. We didn’t work on (1) time travel, (2) teleportation, and (3) antigravity. They are not important problems because we do not have an attack. It’s not the consequence that makes a problem important, it is that you have a reasonable attack.
We can talk productively here about topics like decision theory because we have an attack, a small foothold of sanity (established mostly by Eliezer and Wei) that gives us a firm footing to expand our understanding. As far as I can see, we have no such footholds in politics, or gender relations, or most of those other important topics you listed. I’ve been here for a long time and know that most of our interminable “discussions” of these controversial topics have been completely useless. Our rationality helps us maintain a civil tone, but not actually, you know, make progress.
Human understanding progresses through small problems solved conclusively, once and forever. The first step in any pre-paradigmatic field (like politics) is always the hardest: you need to generate a piece of insight that allows other people to generate new pieces of insight. It’s not a task for our argumentative circuitry, it’s a task for sitting down and thinking really hard. Encouraging wide discussion is the wrong step in the dance. If you don’t have a specific breakthrough, I’d rather we talked about math.
- 21 Jan 2011 11:26 UTC; 8 points) 's comment on Politics is a fact of life by (
I’ve had several conversations that went like this:
Victim: But surely a smart artificial intelligence will be able to tell right from wrong, if we humans can do that?
Me: Forget about the word “intelligence” for a moment. Imagine a machine that looks at all actions in turn, and mechanically chooses the action that leads to producing the greatest number of paperclips, in whichever way possible. With enough computing power and enough knowledge about the outside world, the machine might find a way to convert the whole world into a paperclip factory. The machine will resist any attempts by humans to interfere, because the machine’s goal function doesn’t say anything about humans, only paperclips.
Victim: But such a machine would not be truly intelligent.
Me: Who cares about definitions of words? Humanity can someday find a way to build such a machine, and then we’re all screwed.
Victim: …okay, I see your point. Your machine is not intelligent, but it can be very dangerous because it’s super-efficient.
Me (under my breath): Yeah. That’s actually my definition of “superintelligent”, but you seem to have a concept of “intelligence” that’s entangled with many accidental facts about humans, so let’s not go there.
I think all these worldview enhancements and drugs are no longer a solution to your original problem (not fitting in) and have evolved to a scary problem in itself. You should probably start pushing back.
In the post I tried pretty hard to show the applicability of the techniques to real life, and so did Schelling. Apparently we haven’t succeeded. Maybe some more quotes will tip the scales? Something of a more general nature, not ad hoc trickery?
If one is committed to punish a certain type of behavior when it reaches certain limits, but the limits are not carefully and objectively defined, the party threatened will realize when the time comes to decide whether the threat must be enforced or not, his interest and that of the threatening party will coincide in an attempt to avoid the mutually unpleasant consequences.
Or what do you say to this:
Among the legal privileges of corporations, two that are mentioned in textbooks are the right to sue and the “right” to be sued. Who wants to be sued! But the right to be sued is the power to make a promise: to borrow money, to enter a contract, to do business with someone who might be damaged. If suit does arise, the “right” seems a liability in retrospect; beforehand it was a prerequisite to doing business.
Or this:
If each party agrees to send a million dollars to the Red Cross on condition the other does, each may be tempted to cheat if the other contributes first, and each one’s anticipation of the other’s cheating will inhibit agreement. But if the contribution is divided into successive small contributions, each can try the other’s good faith for a small price. Furthermore, since each can keep the other on short tether to the finish, no one ever need risk more than one small contribution at a time. Finally, this change in the incentive structure itself takes most of the risk out of the initial contribution; the value of established trust is made obviously visible to both.
Or this:
When there are two objects to negotiate, the decision to negotiate them simultaneously or in separate forums or at separate times is by no means neutral to the outcome, particularly when there is a latent extortionate threat that can be exploited only if it can be attached to some ordinary, legitimate, bargaining situation.
I’m not even being particularly picky on which paragraphs to quote. The whole book is like that. To me the main takeaway was not local trickery, but a general way of thinking about conflict situations; I started seeing them everywhere, all the time.
- 31 Jul 2009 19:06 UTC; 1 point) 's comment on An Alternative Approach to AI Cooperation by (
Maybe offtopic, but the “trying too hard to try” part rings very true to me. Been on both sides of it.
The tricky thing about work, I’m realizing more and more, is that you should just work. That’s the whole secret. If instead you start thinking how difficult the work is, or how important to the world, or how you need some self-improvement before you can do the work effectively, these thoughts will slow you down and surprisingly often they’ll be also completely wrong. It always turns out later that your best work wasn’t the one that took the most effort, or felt the most important at the time; you were just having a nose-down busy period, doing a bunch of things, and only the passage of time made clear which of them mattered.
Most of the impact of rape is a made-up self fulfilling prophesy.
The same would apply to cuckoldry.
Upvoted for saying the only thing in this whole thread that makes my inner animal go “aaaaugh I must fight against people who say that”. I didn’t know I had it in me.
There was no interesting, non-emotional takeaway for me from this post. I bet I can find 10 anecdotes where nature prevailed over nurture...
Luke, your posts might become more interesting (to me at least) if you go and beat your head on a hard problem for awhile. I know that sounds like “go have an awful life so you have some interesting stories to tell us”, but hey, that’s how life works :-/ As far as I know, Eliezer’s sequences were the aftermath of a sort of hero’s journey, that’s why they have so many new insights. Just copying the surface pathos won’t get you there.
So there’s this very complicated moment of a group coming together, where enough individuals, for whatever reason, sort of agree that something worthwhile is happening, and the decision they make at that moment is: This is good and must be protected. And at that moment, even if it’s subconscious, you start getting group effects. And the effects that we’ve seen come up over and over and over again in online communities...
The first is sex talk, what he called, in his mid-century prose, “A group met for pairing off.” And what that means is, the group conceives of its purpose as the hosting of flirtatious or salacious talk or emotions passing between pairs of members...
The second basic pattern that Bion detailed: The identification and vilification of external enemies. This is a very common pattern. Anyone who was around the Open Source movement in the mid-Nineties could see this all the time...
The third pattern Bion identified: Religious veneration. The nomination and worship of a religious icon or a set of religious tenets. The religious pattern is, essentially, we have nominated something that’s beyond critique. You can see this pattern on the Internet any day you like...
So these are human patterns that have shown up on the Internet, not because of the software, but because it’s being used by humans. Bion has identified this possibility of groups sandbagging their sophisticated goals with these basic urges. And what he finally came to, in analyzing this tension, is that group structure is necessary. Robert’s Rules of Order are necessary. Constitutions are necessary. Norms, rituals, laws, the whole list of ways that we say, out of the universe of possible behaviors, we’re going to draw a relatively small circle around the acceptable ones.
He said the group structure is necessary to defend the group from itself. Group structure exists to keep a group on target, on track, on message, on charter, whatever. To keep a group focused on its own sophisticated goals and to keep a group from sliding into these basic patterns. Group structure defends the group from the action of its own members.
-- Clay Shirky, “A Group Is Its Own Worst Enemy”
- 7 Oct 2009 12:14 UTC; 8 points) 's comment on The First Step is to Admit That You Have a Problem by (
- 14 Aug 2010 3:58 UTC; 3 points) 's comment on Should I believe what the SIAI claims? by (
Servers take resources to keep up.
No they don’t!!
If you gave me a million scientific articles in PDF form that were previously unavailable on the open web, which could be redistributed without legal problems, then I would host them somewhere and pay for it until the day I die. The benefit to humanity is way bigger than the trivial cost to me, and I also gain some much needed geek karma :-) Are there any LWers who wouldn’t do the same?
just getting out of the house.
...Damn! That’s exactly the kind of vague advice that HughRistik decries. Imagine teaching an extraterrestrial alien to smoke cigarettes.
You: Open the pack.
Alien: (looks at pack in a puzzled way)
You: Just tear it open, man
Alien: (tears pack in half, cigarette bits fly everywhere)
...
You: Put the cigarette in your mouth.
Alien: (stuffs entire cigarette into mouth)
And so on, and so forth. “Get out of the house” is a totally useless piece of advice for the kind of person that needs it. Okay, I’m out of my house right now, what next? You remind me of Alicorn who wouldn’t stop insisting that finding potential dates in your social circle is “easy” if you “just do it”.
(Related: I’ve entertained the idea of suggesting to Alicorn that she apply her superior understanding of women to teach pickup to male students. I imagine her entering the classroom, glancing at the audience composed of actual average guys and going ”...oh, you meant that kind of average? I had no idea such people even existed. Obviously, teaching them to approach women would be disgusting and a gross betrayal of my sex. I’m outta here.”)
- 14 Jul 2010 22:28 UTC; 2 points) 's comment on How to always have interesting conversations by (
Just kidding. It’s called disulfiram, and it was approved by the FDA in 1951.
Cute turnaround + mention of FDA = instant feeling of reading Scott.
LW has helped me a lot. Not in matters of finding the truth; you can be a good researcher without reading LW, as the whole history of science shows. (More disturbingly, you can be a good researcher of QM stuff, read LW, disagree with Eliezer about MWI, have a good chance of being wrong, and not be crippled by that in the least! Huh? Wasn’t it supposed to be all-important to have the right betting odds?) No; for me LW is mostly useful for noticing bullshit and cutting it away from my thoughts. When LW says someone’s wrong, we may or may not be right; but when LW says someone’s saying bullshit, we’re probably right.
I believe that Eliezer has succeeded in creating, and communicating through the Sequences, a valuable technique for seeing through words to their meanings and trying to think correctly about those instead. When you do that, you inevitably notice how much of what you considered to be “meanings” is actually yay/boo reactions, or cached conclusions, or just fine mist that dissolves when you look at it closely. Normal folks think that the question about a tree falling in the forest is kinda useless; nerdy folks suppress their flinch reaction and get confused instead; extra nerdy folks know exactly why the question is useless. Normal folks don’t let politics overtake their mind; concerned folks get into huge flamewars; but we know exactly why this is counterproductive. I liked reading Moldbug before LW. Now I find him… occasionally entertaining, I guess?
Better people than I are already turning this into a sort of martial art. Look at Yvain cutting down ten guys with one swoop, and then try to tell me LW isn’t useful!
- 27 Aug 2012 17:40 UTC; 8 points) 's comment on Stupid Questions Open Thread Round 4 by (
What Thomas Schelling would do. Partly tongue-in-cheek.
The Clumsy Game-Player: agree to the deal, then perform an identical “finger slip” several turns later.
The Lazy Student, The Grieving Student, The Sports Fan: make the deadline for reports a curve instead of a cliff. Each day of delay costs some percentage of the grade.
The Murderous Husband: if you really don’t want these things to happen, make the wife partially responsible for the murder in such cases, by law. (Or the lover, if the husband chooses to murder the wife.)
The Bellicose Dictator: publicly threaten sanctions unless the invading army withdraws immediately. Do this before any negotiations.
The Peyote-Popping Native, The Well-Disguised Atheist: when the native first comes to you, offer to balance out the permission to smoke peyote with some sanction against the Native American church. Then the atheists won’t bother asking for a free lunch.