Regardless of the odds of success and what the optimal course of action actually is, I would be very hard pressed to say that I’m trying to “help humanity die with dignity”. Regardless of what the optimal action should be given that goal, on an emotional level, it’s tantamount to giving up.
Before even getting into the cost/benefit of that attitude, in the worlds where we do make it out alive, I don’t want to look back and see a version of me where that became my goal. I also don’t think that if that was my goal, that I would fight nearly as hard to achieve it. I want a catgirl volcano lair not “dignity”. So when I try to negotiate with my money brain to expend precious calories, the plan had better involve the former, not the latter. I suspect that something similar applies to others.
I don’t want to hear about genre-saviness from the defacto-founder of the community that gave us HPMOR!Harry and the Comet King after he wrote this post. Because it’s so antithetical to the attitude present in those characters and posts like this one.
I also don’t want to hear about second-order effects when, as best as I can tell, the attitude present here is likely to push people towards ineffective doomerism, rather than actually dying with dignity.
So instead, I’m gonna think carefully about my next move, come up with a plan, blast some shonen anime OSTs, and get to work. Then, amongst all the counterfactual worlds, there will be a version of me that gets to look back and know that they faced the end of the world, rose to the challenge, and came out the other end having carved utopia out of the bones of lovecraftian gods.
I think there’s an important point about locus of control and scope. You can imagine someone who, early in life, decides that their life’s work will be to build a time machine, because the value of doing so is immense (turning an otherwise finite universe into an infinite one, for example). As time goes on, they notice being more and more pessimistic about their prospects of doing that, but have some block against giving up on an emotional level. The stakes are too high for doomerism to be entertained!
But I think they overestimated their locus of control when making their plans, and they should have updated as evidence came in. If they reduced the scope of their ambitions, they might switch from plans that are crazy because they have to condition on time travel being possible to plans that are sane (because they can condition on actual reality). Maybe they just invent flying cars instead of time travel, or whatever.
I see this post as saying: “look, people interested in futurism: if you want to live in reality, this is where the battle line actually is. Fight your battles there, don’t send bombing runs behind miles of anti-air defenses and wonder why you don’t seem to be getting any hits.” Yes, knowing the actual state of the battlefield might make people less interested in fighting in the war, but especially for intellectual wars it doesn’t make sense to lie to maintain morale.
[In particular, lies of the form “alignment is easy!” work both to attract alignment researchers and convince AI developers and their supporters that developing AI is good instead of world-ending, because someone else is handling the alignment bit.]
Aside: Regardless of whether the quoted claim is true, it does not seem like a prototypical lie. My read of your meaning is: “If you [the hypothetical person claiming alignment is easy] were an honest reasoner and worked out the consequences of what you know, you would not believe that alignment is easy; thusly has an inner deception blossomed into an outer deception; thus I call your claim a ‘lie.’”
And under that understanding of what you mean, Vaniver, I think yours is not a wholly inappropriate usage, but rather unconventional. In its unconventionality, I think it implies untruths about the intentions of the claimants. (Namely, that they semi-consciously seek to benefit by spreading a claim they know to be false on some level.) In your shoes, I think I would have just called it an “untruth” or “false claim.”
Edit: I now think you might have been talking about EY’s hypothetical questioners who thought it valuable to purposefully deceive about the problem’s difficulty, and not about the typical present-day person who believes alignment is easy?
Edit: I now think you might have been talking about EY’s hypothetical questioners who thought it valuable to purposefully deceive about the problem’s difficulty, and not about the typical present-day person who believes alignment is easy?
“To win any battle, you must fight as if you are already dead.” — Miyamoto Musashi.
I don’t in fact personally know we won’t make it. This may be because I’m more ignorant than Eliezer, or may be because he (or his April first identity, I guess) is overconfident on a model, relative to me; it’s hard to tell.
Regardless, the bit about “don’t get psychologically stuck having-to-(believe/pretend)-it’s-gonna-work seems really sane and healthy to me. Like falling out of an illusion and noticing your feet on the ground. The ground is a more fun and joyful place to live, even when things are higher probability of death than one is used to acting-as-though, in my limited experience. More access to creativity near the ground, I think.
But, yes, I can picture things under the heading “ineffective doomerism” that seem to me like they suck. Like, still trying to live in an ego-constructed illusion of deferral, and this time with “and we die” pasted on it, instead of “and we definitely live via such-and-such a plan.”
I think I have more access to all of my emotional range nearer the ground, but this sentence doesn’t ring true to me.
The ground is a more fun and joyful place to live, even when things are higher probability of death than one is used to acting-as-though, in my limited experience.
As cheesy as it is, this is the correct response. I’m a little disappointed that Eliezer would resort to doomposting like this, but at the same time it’s to be expected from him after some point. The people with remaining energy need to understand his words are also serving a personal therapeautic purpose and press on.
Yeah—I love AI_WAIFU’s comment, but I love the OP too.
To some extent I think these are just different strategies that will work better for different people; both have failure modes, and Eliezer is trying to guard against the failure modes of ‘Fuck That Noise’ (e.g., losing sight of reality), while AI_WAIFU is trying to guard against the failure modes of ‘Try To Die With More Dignity’ (e.g., losing motivation).
My general recommendation to people would be to try different framings / attitudes out and use the ones that empirically work for them personally, rather than trying to have the same lens as everyone else. I’m generally a skeptic of advice, because I think people vary a lot; so I endorse the meta-advice that you should be very picky about which advice you accept, and keep in mind that you’re the world’s leading expert on yourself. (Or at least, you’re in the best position to be that thing.)
Cf. ‘Detach the Grim-o-Meter’ versus ‘Try to Feel the Emotions that Match Reality’. Both are good advice in some contexts, for some people; but I think there’s some risk from taking either strategy too far, especially if you aren’t aware of the other strategy as a viable option.
Please correct me if I am wrong, but a huge difference between Eliezer’s post and AI_WAIFU’s comment is that Eliezer’s post is informed by conversations with dozens of people about the problem.
I interpreted AI_WAIFU as pushing back against a psychological claim (‘X is the best attitude for mental clarity, motivation, etc.’), not as pushing back against a AI-related claim like P(doom). Are you interpreting them as disagreeing about P(doom)? (If not, then I don’t understand your comment.)
If (counterfactually) they had been arguing about P(doom), I’d say: I don’t know AI_WAIFU’s level of background. I have a very high opinion of Eliezer’s thinking about AI (though keep in mind that I’m a co-worker of his), but EY is still some guy who can be wrong about things, and I’m interested to hear counter-arguments against things like P(doom). AGI forecasting and alignment are messy, pre-paradigmatic fields, so I think it’s easier for field founders and authorities to get stuff wrong than it would be in a normal scientific field.
The specific claim that Eliezer’s P(doom) is “informed by conversations with dozens of people about the problem” (if that’s what you were claiming) seems off to me. Like, it may be technically true under some interpretation, but (a) I think of Eliezer’s views as primarily based on his own models, (b) I’d tentatively guess those models are much more based on things like ‘reading textbooks’ and ‘thinking things through himself’ than on ‘insights gleaned during back-and-forth discussions with other people’, and (c) most people working full-time on AI alignment have far lower P(doom) than Eliezer.
Sorry for the lack of clarity. I share Eliezer’s pessimism about the global situation (caused by rapid progress in AI). All I meant is that I see signs in his writings that over the last 15 years Eliezer has spent many hours trying to help at least a dozen different people become effective at trying to improve the horrible situation we are currently in. That work experience makes me pay much greater attention to him on the subject at hand than someone I know nothing about.
Ah, I see. I think Eliezer has lots of relevant experience and good insights, but I still wouldn’t currently recommend the ‘Death with Dignity’ framing to everyone doing good longtermist work, because I just expect different people’s minds to work very differently.
Assuming this is correct (certainly it is of Eliezer, though I don’t know AI_WAIFU’s background and perhaps they have had similar conversations), does it matter? WAIFU’s point is that we should continue trying as a matter of our terminal values; that’s not something that can be wrong due to the problem being difficult.
I agree, but do not perceive Eliezer as having stopped trying or as advising others to stop trying, er, except of course for the last section of this post (“Q6: . . . All of this is just an April Fool’s joke, right?”) but that is IMHO addressed to a small fraction of his audience.
I don’t want to speak for him (especially when he’s free to clarify himself far better than we could do for him!), but dying with dignity conveys an attitude that might be incompatible with actually winning. Maybe not; sometimes abandoning the constraint that you have to see a path to victory makes it easier to do the best you can. But it feels concerning on an instinctive level.
I think I’m more motivated by the thought that I am going to die soon, any children I might have in the future will die soon, my family, my friends, and their children are going to die soon, and any QALYs I think I’m buying are around 40% as valuable as I thought, more than undoing the income tax deduction I get for them.
It seems like wrangling my ADHD brain into looking for way to prevent catastrophe could be more worthwhile than working a high-paid job I can currently hyper-focus on (and probably more virtuous, too), unless I find that the probability of success is literally 0% despite what I think I know about Bayesian reasoning, in which case I’ll probably go into art or something.
Do not go gentle into that good night,
Old age should burn and rave at close of day;
Rage, rage against the dying of the light.
Though wise men at their end know dark is right,
Because their words had forked no lightning they
Do not go gentle into that good night.
Good men, the last wave by, crying how bright
Their frail deeds might have danced in a green bay,
Rage, rage against the dying of the light.
Wild men who caught and sang the sun in flight,
And learn, too late, they grieved it on its way,
Do not go gentle into that good night.
Grave men, near death, who see with blinding sight
Blind eyes could blaze like meteors and be gay,
Rage, rage against the dying of the light.
And you, my father, there on the sad height,
Curse, bless, me now with your fierce tears, I pray.
Do not go gentle into that good night.
Rage, rage against the dying of the light.
I totally empathize with Eliezer, and I’m afraid that I might be similarly burned out if I had been trying this for as long.
Most fictional characters are optimised to make for entertaining stories, hence why “generalizing from fictional evidence” is usually a failure-mode. The HPMOR Harry and the Comet King were optimized by two rationalists as examples of rationalist heroes — and are active in allegorical situations engineered to say something that rationalists would find to be “of worth” about real world problems.
They are appealing precisely because they encode assumptions about what a real-world, rationalist “hero” ought to be like. Or at least, that’s the hope. So, they can be pointed to as “theses” about the real world by Yudkowsky and Alexander, no different from blog posts that happen to be written as allegorical stories, and if people found the ideas encoded in those characters more convincing than the ideas encoded in the present April Fools’ Day post, that’s fair enough.
Not necessarily correct on the object-level, but, if it’s wrong, it’s a different kind of error from garden-variety “generalizing from fictional evidence”.
As fictional characters popular among humans, what attitude is present in them is evidence for what sort of attitude humans like to see or inhabit. As author of those characters, Yudkowsky should be aware of this mechanism. And empirically, people with accurate beliefs and positive attitudes outperform people with accurate beliefs and negative attitudes. It seems plausible Yudkowsky is aware of this as well.
“Death with dignity” reads as an unnecessarily negative attitude to accompany the near-certainty of doom. Heroism, maximum probability of catgirls, or even just raw log-odds-of-survival seem like they would be more motivating than dignity without sacrificing accuracy.
Like, just substitute all instances of ‘dignity’ in the OP with ‘heroism’ and naively I would expect this post to have a better impact(/be more dignified/be more heroic), except insofar it might give a less accurate impression of Yudkowsky’s mood. But few people have actually engaged with him on that front.
That’s fair; thanks for the feedback! I’ll tone down the gallows humor on future comments; gotta keep in mind that tone of voice doesn’t come across.
BTW a money brain would arise out of, e.g., a merchant caste in a static medieval society after many millennia. Much better than a monkey brain, and more capable of solving alignment!
That’s great and all, but with all due respect:
Fuck. That. Noise.
Regardless of the odds of success and what the optimal course of action actually is, I would be very hard pressed to say that I’m trying to “help humanity die with dignity”. Regardless of what the optimal action should be given that goal, on an emotional level, it’s tantamount to giving up.
Before even getting into the cost/benefit of that attitude, in the worlds where we do make it out alive, I don’t want to look back and see a version of me where that became my goal. I also don’t think that if that was my goal, that I would fight nearly as hard to achieve it. I want a catgirl volcano lair not “dignity”. So when I try to negotiate with my money brain to expend precious calories, the plan had better involve the former, not the latter. I suspect that something similar applies to others.
I don’t want to hear about genre-saviness from the defacto-founder of the community that gave us HPMOR!Harry and the Comet King after he wrote this post. Because it’s so antithetical to the attitude present in those characters and posts like this one.
I also don’t want to hear about second-order effects when, as best as I can tell, the attitude present here is likely to push people towards ineffective doomerism, rather than actually dying with dignity.
So instead, I’m gonna think carefully about my next move, come up with a plan, blast some shonen anime OSTs, and get to work. Then, amongst all the counterfactual worlds, there will be a version of me that gets to look back and know that they faced the end of the world, rose to the challenge, and came out the other end having carved utopia out of the bones of lovecraftian gods.
I think there’s an important point about locus of control and scope. You can imagine someone who, early in life, decides that their life’s work will be to build a time machine, because the value of doing so is immense (turning an otherwise finite universe into an infinite one, for example). As time goes on, they notice being more and more pessimistic about their prospects of doing that, but have some block against giving up on an emotional level. The stakes are too high for doomerism to be entertained!
But I think they overestimated their locus of control when making their plans, and they should have updated as evidence came in. If they reduced the scope of their ambitions, they might switch from plans that are crazy because they have to condition on time travel being possible to plans that are sane (because they can condition on actual reality). Maybe they just invent flying cars instead of time travel, or whatever.
I see this post as saying: “look, people interested in futurism: if you want to live in reality, this is where the battle line actually is. Fight your battles there, don’t send bombing runs behind miles of anti-air defenses and wonder why you don’t seem to be getting any hits.” Yes, knowing the actual state of the battlefield might make people less interested in fighting in the war, but especially for intellectual wars it doesn’t make sense to lie to maintain morale.
[In particular, lies of the form “alignment is easy!” work both to attract alignment researchers and convince AI developers and their supporters that developing AI is good instead of world-ending, because someone else is handling the alignment bit.]
Aside: Regardless of whether the quoted claim is true, it does not seem like a prototypical lie. My read of your meaning is: “If you [the hypothetical person claiming alignment is easy] were an honest reasoner and worked out the consequences of what you know, you would not believe that alignment is easy; thusly has an inner deception blossomed into an outer deception; thus I call your claim a ‘lie.’”
And under that understanding of what you mean, Vaniver, I think yours is not a wholly inappropriate usage, but rather unconventional. In its unconventionality, I think it implies untruths about the intentions of the claimants. (Namely, that they semi-consciously seek to benefit by spreading a claim they know to be false on some level.) In your shoes, I think I would have just called it an “untruth” or “false claim.”
Edit: I now think you might have been talking about EY’s hypothetical questioners who thought it valuable to purposefully deceive about the problem’s difficulty, and not about the typical present-day person who believes alignment is easy?
That is what I was responding to.
“To win any battle, you must fight as if you are already dead.” — Miyamoto Musashi.
I don’t in fact personally know we won’t make it. This may be because I’m more ignorant than Eliezer, or may be because he (or his April first identity, I guess) is overconfident on a model, relative to me; it’s hard to tell.
Regardless, the bit about “don’t get psychologically stuck having-to-(believe/pretend)-it’s-gonna-work seems really sane and healthy to me. Like falling out of an illusion and noticing your feet on the ground. The ground is a more fun and joyful place to live, even when things are higher probability of death than one is used to acting-as-though, in my limited experience. More access to creativity near the ground, I think.
But, yes, I can picture things under the heading “ineffective doomerism” that seem to me like they suck. Like, still trying to live in an ego-constructed illusion of deferral, and this time with “and we die” pasted on it, instead of “and we definitely live via such-and-such a plan.”
I think I have more access to all of my emotional range nearer the ground, but this sentence doesn’t ring true to me.
Hm. It rings true to me, but there have been periods of my life where it has been false.
As cheesy as it is, this is the correct response. I’m a little disappointed that Eliezer would resort to doomposting like this, but at the same time it’s to be expected from him after some point. The people with remaining energy need to understand his words are also serving a personal therapeautic purpose and press on.
Some people can think there’s next to no chance and yet go out swinging. I plan to, if I reach the point of feeling hopeless.
Yeah—I love AI_WAIFU’s comment, but I love the OP too.
To some extent I think these are just different strategies that will work better for different people; both have failure modes, and Eliezer is trying to guard against the failure modes of ‘Fuck That Noise’ (e.g., losing sight of reality), while AI_WAIFU is trying to guard against the failure modes of ‘Try To Die With More Dignity’ (e.g., losing motivation).
My general recommendation to people would be to try different framings / attitudes out and use the ones that empirically work for them personally, rather than trying to have the same lens as everyone else. I’m generally a skeptic of advice, because I think people vary a lot; so I endorse the meta-advice that you should be very picky about which advice you accept, and keep in mind that you’re the world’s leading expert on yourself. (Or at least, you’re in the best position to be that thing.)
Cf. ‘Detach the Grim-o-Meter’ versus ‘Try to Feel the Emotions that Match Reality’. Both are good advice in some contexts, for some people; but I think there’s some risk from taking either strategy too far, especially if you aren’t aware of the other strategy as a viable option.
Please correct me if I am wrong, but a huge difference between Eliezer’s post and AI_WAIFU’s comment is that Eliezer’s post is informed by conversations with dozens of people about the problem.
I interpreted AI_WAIFU as pushing back against a psychological claim (‘X is the best attitude for mental clarity, motivation, etc.’), not as pushing back against a AI-related claim like P(doom). Are you interpreting them as disagreeing about P(doom)? (If not, then I don’t understand your comment.)
If (counterfactually) they had been arguing about P(doom), I’d say: I don’t know AI_WAIFU’s level of background. I have a very high opinion of Eliezer’s thinking about AI (though keep in mind that I’m a co-worker of his), but EY is still some guy who can be wrong about things, and I’m interested to hear counter-arguments against things like P(doom). AGI forecasting and alignment are messy, pre-paradigmatic fields, so I think it’s easier for field founders and authorities to get stuff wrong than it would be in a normal scientific field.
The specific claim that Eliezer’s P(doom) is “informed by conversations with dozens of people about the problem” (if that’s what you were claiming) seems off to me. Like, it may be technically true under some interpretation, but (a) I think of Eliezer’s views as primarily based on his own models, (b) I’d tentatively guess those models are much more based on things like ‘reading textbooks’ and ‘thinking things through himself’ than on ‘insights gleaned during back-and-forth discussions with other people’, and (c) most people working full-time on AI alignment have far lower P(doom) than Eliezer.
Sorry for the lack of clarity. I share Eliezer’s pessimism about the global situation (caused by rapid progress in AI). All I meant is that I see signs in his writings that over the last 15 years Eliezer has spent many hours trying to help at least a dozen different people become effective at trying to improve the horrible situation we are currently in. That work experience makes me pay much greater attention to him on the subject at hand than someone I know nothing about.
Ah, I see. I think Eliezer has lots of relevant experience and good insights, but I still wouldn’t currently recommend the ‘Death with Dignity’ framing to everyone doing good longtermist work, because I just expect different people’s minds to work very differently.
Assuming this is correct (certainly it is of Eliezer, though I don’t know AI_WAIFU’s background and perhaps they have had similar conversations), does it matter? WAIFU’s point is that we should continue trying as a matter of our terminal values; that’s not something that can be wrong due to the problem being difficult.
I agree, but do not perceive Eliezer as having stopped trying or as advising others to stop trying, er, except of course for the last section of this post (“Q6: . . . All of this is just an April Fool’s joke, right?”) but that is IMHO addressed to a small fraction of his audience.
I don’t want to speak for him (especially when he’s free to clarify himself far better than we could do for him!), but dying with dignity conveys an attitude that might be incompatible with actually winning. Maybe not; sometimes abandoning the constraint that you have to see a path to victory makes it easier to do the best you can. But it feels concerning on an instinctive level.
In my experience, most people cannot.
I think both emotions are helpful at motivating me.
I think I’m more motivated by the thought that I am going to die soon, any children I might have in the future will die soon, my family, my friends, and their children are going to die soon, and any QALYs I think I’m buying are around 40% as valuable as I thought, more than undoing the income tax deduction I get for them.
It seems like wrangling my ADHD brain into looking for way to prevent catastrophe could be more worthwhile than working a high-paid job I can currently hyper-focus on (and probably more virtuous, too), unless I find that the probability of success is literally 0% despite what I think I know about Bayesian reasoning, in which case I’ll probably go into art or something.
Agreed. Also here’s the poem that goes with that comment:
I totally empathize with Eliezer, and I’m afraid that I might be similarly burned out if I had been trying this for as long.
But that’s not who I want to be. I want to be Harry who builds a rocket to escape Azkaban, the little girl that faces the meteor with a baseball bat, and the general who empties his guns into the sky against another meteor (minus all his racism and shit).
I bet I won’t always have the strength for that – but that’s the goal.
Makes me think of the following quote. I’m not sure how much I agree with or endorse it, but it’s something to think about.
Exquisitely based
Harry Potter and the Comet King have access to magic; we don’t.
… is the obvious response, but the correct response is actually:
Harry Potter and the Comet King don’t exist, so what attitude is present in those characters is irrelevant to the question of what attitude we, in reality, ought to have.
Most fictional characters are optimised to make for entertaining stories, hence why “generalizing from fictional evidence” is usually a failure-mode. The HPMOR Harry and the Comet King were optimized by two rationalists as examples of rationalist heroes — and are active in allegorical situations engineered to say something that rationalists would find to be “of worth” about real world problems.
They are appealing precisely because they encode assumptions about what a real-world, rationalist “hero” ought to be like. Or at least, that’s the hope. So, they can be pointed to as “theses” about the real world by Yudkowsky and Alexander, no different from blog posts that happen to be written as allegorical stories, and if people found the ideas encoded in those characters more convincing than the ideas encoded in the present April Fools’ Day post, that’s fair enough.
Not necessarily correct on the object-level, but, if it’s wrong, it’s a different kind of error from garden-variety “generalizing from fictional evidence”.
+1
As fictional characters popular among humans, what attitude is present in them is evidence for what sort of attitude humans like to see or inhabit. As author of those characters, Yudkowsky should be aware of this mechanism. And empirically, people with accurate beliefs and positive attitudes outperform people with accurate beliefs and negative attitudes. It seems plausible Yudkowsky is aware of this as well.
“Death with dignity” reads as an unnecessarily negative attitude to accompany the near-certainty of doom. Heroism, maximum probability of catgirls, or even just raw log-odds-of-survival seem like they would be more motivating than dignity without sacrificing accuracy.
Like, just substitute all instances of ‘dignity’ in the OP with ‘heroism’ and naively I would expect this post to have a better impact(/be more dignified/be more heroic), except insofar it might give a less accurate impression of Yudkowsky’s mood. But few people have actually engaged with him on that front.
You have a money brain? That’s awesome, most of us only have monkey brains! 🙂
Why the downboats? People new to LW jargon probably wouldn’t realize “money brain” is a typo.
Seemed like a bit of a rude way to let someone know they had a typo, I would have just gone with “Typo: money brain should be monkey brain”.
That’s fair; thanks for the feedback! I’ll tone down the gallows humor on future comments; gotta keep in mind that tone of voice doesn’t come across.
BTW a money brain would arise out of, e.g., a merchant caste in a static medieval society after many millennia. Much better than a monkey brain, and more capable of solving alignment!
Here we see irrationality for what it is: a rational survival mechanism.