At a glance, I think this works, and it’s a neat approach. I have doubts, though.
The impossibility of explaining the theory behind this to random SV CEOs or military leaders… is not one of them. The human culture had always contained many shards of LDT-style thinking, the concept of “honour” chief among them. To explain it, you don’t actually need to front-load exposition about decision theories and acausal trade – you can just re-use said LDT-shards, and feed them the idea in an immediately intuitive format.
I’m not entirely sure how that would look like, and it’s not a trivial re-framing problem. But I think it’s very doable with some crafty memetic engineering.
My first concern is that we might not actually have the time. While proliferating this idea (in the sense of “This Is What You Do If You Have AGI”) is doable, that’d still take some time. You’d need to split it into a set of five-word messages, and parcel them out over the years. I think you’d said your timeline is 0-5 years; that’s IMO definitely not enough.
Based on the latest developments (Gemini, what Q* is supposed to be, both underwhelming), I think we have a fair bit longer (like, a decade-ish). Might be enough if we start yesterday.
My second concern is… more vague, but I feel like this is still being too optimistic about the human nature. Sure, maybe it’d work for the current crop of major-AI-Lab CEOs. But in a lot of situations (e. g., acute xenophobia), I think the preference ordering goes “I win” > “they lose” > “a compromise”, such that they would prefer an all-or-nothing gamble to a measured split of the gains. (Like, it’s almost Copenhagen Ethics-ish? It feels utterly repulsive to contribute to your enemy’s happiness, such that you’d rather either eradicate them or be eradicated, no matter how self-destructive that is?)
At that point, I may be being too cynical, though. I also might feel differently if I were staring at the version of this pitch already re-framed into intuitive terms.
Due to my timelines being this short, I’m hopeful that convincing just “the current crop of major-AI-Lab CEOs” might actually be enough to buy us the bulk of time that something like this could buy.
At a glance, I think this works, and it’s a neat approach. I have doubts, though.
The impossibility of explaining the theory behind this to random SV CEOs or military leaders… is not one of them. The human culture had always contained many shards of LDT-style thinking, the concept of “honour” chief among them. To explain it, you don’t actually need to front-load exposition about decision theories and acausal trade – you can just re-use said LDT-shards, and feed them the idea in an immediately intuitive format.
I’m not entirely sure how that would look like, and it’s not a trivial re-framing problem. But I think it’s very doable with some crafty memetic engineering.
My first concern is that we might not actually have the time. While proliferating this idea (in the sense of “This Is What You Do If You Have AGI”) is doable, that’d still take some time. You’d need to split it into a set of five-word messages, and parcel them out over the years. I think you’d said your timeline is 0-5 years; that’s IMO definitely not enough.
Based on the latest developments (Gemini, what Q* is supposed to be, both underwhelming), I think we have a fair bit longer (like, a decade-ish). Might be enough if we start yesterday.
My second concern is… more vague, but I feel like this is still being too optimistic about the human nature. Sure, maybe it’d work for the current crop of major-AI-Lab CEOs. But in a lot of situations (e. g., acute xenophobia), I think the preference ordering goes “I win” > “they lose” > “a compromise”, such that they would prefer an all-or-nothing gamble to a measured split of the gains. (Like, it’s almost Copenhagen Ethics-ish? It feels utterly repulsive to contribute to your enemy’s happiness, such that you’d rather either eradicate them or be eradicated, no matter how self-destructive that is?)
At that point, I may be being too cynical, though. I also might feel differently if I were staring at the version of this pitch already re-framed into intuitive terms.
Due to my timelines being this short, I’m hopeful that convincing just “the current crop of major-AI-Lab CEOs” might actually be enough to buy us the bulk of time that something like this could buy.