On your first, I agree and I think your phrasing is better than mine was in section 5.1 of the full post, although I also focused on the dangers of RSI/creating ASI from AGI. Framing it as needing defense to be dominant in all domains (without universal surveillance, which is sort of more like offense) is a very good way to think about the problem with d/acc as a long term strategy. It doesn’t help that most areas actually seem offense-dominant; but even if I’m mistaken about that, you’d need ALL domains of confict to be defense-dominant.
On the others:
1) I don’t think business dynamics are relevant? But yes I do agree that viciousness does often help gain power. But I must note that it also often doesn’t. Power is often accumulated by those who are perceived as trustworthy and cooperative; that’s how they get broad support from many people to gain power. And corporate structures involve a lot more personal knowledge than the democratic let alone undemocratic government structures.
2) I don’t think mating instincts are a distinct case from the analysis I make. If the emperor isn’t highly sadistic, he’d rather have “mates” who like and respect him. So it follows the same empathy-minus-sadism balance as other areas of life. There are a lot of drives and I didn’t try to cover them individually, even in the long version.
3) I agree that good people might fail to take over even when they could do it bloodlessly. But once they have an AGI to tell them “so yeah if you don’t take over this is probably gonna go much worse and maybe everyone dies or gets a permanent dystopia instead of a glorious future” I think they’ll get over their deontological hangups and do the sensible thing and save the future. At least to the extent their AGI is smart enough to be reliable on that judgment call.
Thanks for engaging!
On your first, I agree and I think your phrasing is better than mine was in section 5.1 of the full post, although I also focused on the dangers of RSI/creating ASI from AGI. Framing it as needing defense to be dominant in all domains (without universal surveillance, which is sort of more like offense) is a very good way to think about the problem with d/acc as a long term strategy. It doesn’t help that most areas actually seem offense-dominant; but even if I’m mistaken about that, you’d need ALL domains of confict to be defense-dominant.
On the others:
1) I don’t think business dynamics are relevant? But yes I do agree that viciousness does often help gain power. But I must note that it also often doesn’t. Power is often accumulated by those who are perceived as trustworthy and cooperative; that’s how they get broad support from many people to gain power. And corporate structures involve a lot more personal knowledge than the democratic let alone undemocratic government structures.
2) I don’t think mating instincts are a distinct case from the analysis I make. If the emperor isn’t highly sadistic, he’d rather have “mates” who like and respect him. So it follows the same empathy-minus-sadism balance as other areas of life. There are a lot of drives and I didn’t try to cover them individually, even in the long version.
3) I agree that good people might fail to take over even when they could do it bloodlessly. But once they have an AGI to tell them “so yeah if you don’t take over this is probably gonna go much worse and maybe everyone dies or gets a permanent dystopia instead of a glorious future” I think they’ll get over their deontological hangups and do the sensible thing and save the future. At least to the extent their AGI is smart enough to be reliable on that judgment call.