I kind of doubt that leaders at big labs would self-identify as being motivated by anything like Eliezer’s notion of heroic responsibility. If any do self-identify that way though, they’re either doing it wrong or misunderstanding. Eliezer has written tons of stuff about the need to respect deontology and also think about all of the actual consequences of your actions, even (especially when) the stakes are high:
The critical question here is: what happens if the plot successfully places the two of them in an epistemic Cooperation-Defection Dilemma, where rather than the two of them just having different goals, Carissa believes that he is mistaken about what happens...
In this case, Carissa could end up believing that to play ‘Defect’ against him would be to serve even his own goals, better than her Cooperating would serve them. Betraying him might seem like a friendly act, an act of aid.
If he commits to a drastic action he will estimate that actual victory lies at the end of it, and his desperation and sacrifice will not have figured into that estimation process as positive factors. His deontology is not for sale at the price point of failure.
Starting an AI lab in order to join a doomed race to superintelligence, and then engaging in a bunch of mundane squabbles for corporate control, seems like exactly the opposite of the sentiment here:
For Albus Dumbledore, as for her, the rule in extremis was to decide what was the right thing to do, and do it no matter the cost to yourself. Even if it meant breaking your bounds, or changing your role, or letting go of your picture of yourself. That was the last resort of Gryffindor.
It also seemingly justifies or obligates Sam Altman to fight back when the OpenAI board tried to fire him, if he believed the board was interfering with his mission.
In general, it seems perfectly fine and normal for a founder-CEO to fight back against a board ouster—no need to bring heroic responsibility into it. Of course, all parties including the CEO and the board should stick to legal / above-board / ethical means of “fighting back”, but if there’s a genuine disagreement between the board and the CEO on how to best serve shareholder interests (or humanity’s interests, for a non-profit), why wouldn’t both sides vigorously defend their own positions and power?
Perhaps the intended reading of your example is that heroic responsibility would obligate or justify underhanded tactics to win control, when the dispute has existential consequences. But I think that’s a misunderstanding of the actual concept. Ordinary self-confidence and agency obligate you to defend your own interests / beliefs / power, and heroic responsibility says that you’re obligated to win without stepping outside the bounds of deontology or slipping into invalid / motivated reasoning.
I kind of doubt that leaders at big labs would self-identify as being motivated by anything like Eliezer’s notion of heroic responsibility. If any do self-identify that way though, they’re either doing it wrong or misunderstanding. Eliezer has written tons of stuff about the need to respect deontology and also think about all of the actual consequences of your actions, even (especially when) the stakes are high:
(https://glowfic.com/replies/1874768#reply-1874768)
(https://glowfic.com/replies/1940939#reply-1940939)
Starting an AI lab in order to join a doomed race to superintelligence, and then engaging in a bunch of mundane squabbles for corporate control, seems like exactly the opposite of the sentiment here:
(https://hpmor.com/chapter/93)
Also, re this example:
In general, it seems perfectly fine and normal for a founder-CEO to fight back against a board ouster—no need to bring heroic responsibility into it. Of course, all parties including the CEO and the board should stick to legal / above-board / ethical means of “fighting back”, but if there’s a genuine disagreement between the board and the CEO on how to best serve shareholder interests (or humanity’s interests, for a non-profit), why wouldn’t both sides vigorously defend their own positions and power?
Perhaps the intended reading of your example is that heroic responsibility would obligate or justify underhanded tactics to win control, when the dispute has existential consequences. But I think that’s a misunderstanding of the actual concept. Ordinary self-confidence and agency obligate you to defend your own interests / beliefs / power, and heroic responsibility says that you’re obligated to win without stepping outside the bounds of deontology or slipping into invalid / motivated reasoning.