trying to characterize this as some kind of “Nate, Eliezer, Robby are defecting on other people trying to be purely cooperative” seems absurd to me. I am really confused what is going on here.
Everything makes sense when you meditate on how the line between “cooperation” and “defection” isn’t in the territory; it’s a computed concept that agents in a variable-sum game have every incentive to “disagree” (actually, fight) about.
Consider the Nash demand game. Two players name a number between 0 and 100. If the sum is less than or equal to 100, you get the number you named as a percentage of the pie; if the sum exceeds 100, the pie is destroyed. There’s no unique Nash equilibrium. It’s stable if Player 1 says 50 and Player 2 says 50, but it’s also stable if Player 1 says 35 and Player 2 says 65 (or generally n and 100 − n, respectively).
The secret is that there are no natural units of pie (or, equivalently, how much pie everyone “deserves”). Everyone thinks that they’re being “cooperative” and that their partners are “defecting”, because they’re counting the pie differently: Player 1 thinks their slice is 35%, but Player 2 thinks the same physical slice is 65%.
If you don’t think your partner is treating you fairly, your leverage is to threaten to destroy surplus unless they treat you better. That’s what Alexander is doing when he says, “I would like to support it with praxis, but right now I feel very conflicted about this”. He’s saying, “You’d better give me a bigger slice, Player 1, or I’ll destroy some of the pie.”
That’s also what your brain is doing when you say you don’t want to work on this anymore. Scott doesn’t want you to quit! (Partially because he values Lightcone’s work, and partially because it would look bad for him if you can publicly blame your burnout on him.) Crucially, your brain knows this. By threatening to quit in frustration, you can probably get Scott to apologize and give your arguments a fairer hearing, whereas in the absence of the threat, he has every incentive to keep being motivatedly dumb from your perspective.
You have a strong hand here! The only risk is if your counterparties don’t think you’d ever actually quit and start calling your bluff. In this case, we know Scott is a pushover and will almost certainly fold. But if you ever face stronger-willed counterparties, you might need to shore up the credibility of your threat: conspicuously going on vacation for a week to think it over will get taken more seriously than an “I don’t know if I want to do this anymore” comment.
(Sorry, maybe you already knew all that, but weren’t articulating it because it’s not part of the game? I don’t think I’m worsening your position that much by saying it out loud; we know that Scott knows this stuff.)
That’s also what your brain is doing when you say you don’t want to work on this anymore. Scott doesn’t want you to quit! (Partially because he values Lightcone’s work, and partially because it would look bad for him if you can publicly blame your burnout on him.) Crucially, your brain knows this.
Man, I really wish this was the case, and it’s non-zero of what is going on, but the vast majority of what I am expressing with my (genuine) desire to quit is the stress and frustration associated with the gaslighting, which is one level more abstract than the issue you talk about.
Like yes, there is a threat here being like “for fuck’s sake, stop gaslighting or I am genuinely going to blow up my part of the pie”, but it’s not actually about the object level, and I don’t actually have much of any genuine hope of that working in the same way one might expect from a negotiation tactic.
I am just genuinely actually very tired, and Scott changing his mind on this and going “oh yeah, actually you are right” actually wouldn’t do much to make me want to not quit, because it wouldn’t address the continuous gaslighting where every time anyone tries to talk about any of the adversarial dynamics, they immediately get told this is all made up and get repeated “I haven’t seen EAs (other than SBF) do a lot of lying, equivocating, or even being particularly shy about their beliefs” and “everyone is being honest all the time and actually it’s just you who is lying right now and always”.
Yeah, the frustrating part is almost always on a meta level. I think Zack’s point about “No natural units of pie” applies to the gaslighting issue as well though. Asserting one’s viewpoint means asserting it as truth which invalidates differing perspectives. “I disagree, you contradict, he gaslights”.
It’s difficult because sometimes the gas lights really don’t seem to be dimming, and sometimes that perception is downstream of some motivated thinking because I really don’t want to believe we’re running out of oil already, dammit. And so the result is simultaneously kinda an honest statement of perspective (at least, as honest as these tend to get) while also being a (not-necessarily-consciously) motivated action pushing people to disregard their own senses. And then we have to decide how to judge this mess of bias and honesty, and if we don’t judge such that the product after a round trip of perceiving C/D and responding accordingly we get more C than last time… shit’s fucked. And without objective units of pie that people can agree on when judging who was in the wrong.
So like… am I trying to gaslight people into questioning their own sanity so they accept what I want them to accept, or am I just flinching away from what scares me, like we all do? Both, and the question of whether I deserve the leniency and empathy is a difficult one, because what are the units of this pie and where’s the objective cutoff? And because our tolerance for further bullshit tends to diminish after accumulating bullshit, so it gets even more difficult to get back to the other side of criticality.
Everything makes sense when you meditate on how the line between “cooperation” and “defection” isn’t in the territory; it’s a computed concept that agents in a variable-sum game have every incentive to “disagree” (actually, fight) about.
Consider the Nash demand game. Two players name a number between 0 and 100. If the sum is less than or equal to 100, you get the number you named as a percentage of the pie; if the sum exceeds 100, the pie is destroyed. There’s no unique Nash equilibrium. It’s stable if Player 1 says 50 and Player 2 says 50, but it’s also stable if Player 1 says 35 and Player 2 says 65 (or generally n and 100 − n, respectively).
The secret is that there are no natural units of pie (or, equivalently, how much pie everyone “deserves”). Everyone thinks that they’re being “cooperative” and that their partners are “defecting”, because they’re counting the pie differently: Player 1 thinks their slice is 35%, but Player 2 thinks the same physical slice is 65%.
If you don’t think your partner is treating you fairly, your leverage is to threaten to destroy surplus unless they treat you better. That’s what Alexander is doing when he says, “I would like to support it with praxis, but right now I feel very conflicted about this”. He’s saying, “You’d better give me a bigger slice, Player 1, or I’ll destroy some of the pie.”
That’s also what your brain is doing when you say you don’t want to work on this anymore. Scott doesn’t want you to quit! (Partially because he values Lightcone’s work, and partially because it would look bad for him if you can publicly blame your burnout on him.) Crucially, your brain knows this. By threatening to quit in frustration, you can probably get Scott to apologize and give your arguments a fairer hearing, whereas in the absence of the threat, he has every incentive to keep being motivatedly dumb from your perspective.
You have a strong hand here! The only risk is if your counterparties don’t think you’d ever actually quit and start calling your bluff. In this case, we know Scott is a pushover and will almost certainly fold. But if you ever face stronger-willed counterparties, you might need to shore up the credibility of your threat: conspicuously going on vacation for a week to think it over will get taken more seriously than an “I don’t know if I want to do this anymore” comment.
(Sorry, maybe you already knew all that, but weren’t articulating it because it’s not part of the game? I don’t think I’m worsening your position that much by saying it out loud; we know that Scott knows this stuff.)
Man, I really wish this was the case, and it’s non-zero of what is going on, but the vast majority of what I am expressing with my (genuine) desire to quit is the stress and frustration associated with the gaslighting, which is one level more abstract than the issue you talk about.
Like yes, there is a threat here being like “for fuck’s sake, stop gaslighting or I am genuinely going to blow up my part of the pie”, but it’s not actually about the object level, and I don’t actually have much of any genuine hope of that working in the same way one might expect from a negotiation tactic.
I am just genuinely actually very tired, and Scott changing his mind on this and going “oh yeah, actually you are right” actually wouldn’t do much to make me want to not quit, because it wouldn’t address the continuous gaslighting where every time anyone tries to talk about any of the adversarial dynamics, they immediately get told this is all made up and get repeated “I haven’t seen EAs (other than SBF) do a lot of lying, equivocating, or even being particularly shy about their beliefs” and “everyone is being honest all the time and actually it’s just you who is lying right now and always”.
Yeah, the frustrating part is almost always on a meta level. I think Zack’s point about “No natural units of pie” applies to the gaslighting issue as well though. Asserting one’s viewpoint means asserting it as truth which invalidates differing perspectives. “I disagree, you contradict, he gaslights”.
It’s difficult because sometimes the gas lights really don’t seem to be dimming, and sometimes that perception is downstream of some motivated thinking because I really don’t want to believe we’re running out of oil already, dammit. And so the result is simultaneously kinda an honest statement of perspective (at least, as honest as these tend to get) while also being a (not-necessarily-consciously) motivated action pushing people to disregard their own senses. And then we have to decide how to judge this mess of bias and honesty, and if we don’t judge such that the product after a round trip of perceiving C/D and responding accordingly we get more C than last time… shit’s fucked. And without objective units of pie that people can agree on when judging who was in the wrong.
So like… am I trying to gaslight people into questioning their own sanity so they accept what I want them to accept, or am I just flinching away from what scares me, like we all do? Both, and the question of whether I deserve the leniency and empathy is a difficult one, because what are the units of this pie and where’s the objective cutoff? And because our tolerance for further bullshit tends to diminish after accumulating bullshit, so it gets even more difficult to get back to the other side of criticality.