It’s a good point, re: some of the gap being that it’s hard to concretely visualize the world in which AGI isn’t built. And also about the “we” being part of the lack of concreteness.
I suspect there’re lots of kinds of ethical heuristics that’re supposed to interweave, and that some are supposed to be more like “checksums” (indicators everyone can use in an embodied way to see whether there’s a problem, even though they don’t say how to address it if there is a problem), and others are supposed to be more concrete.
For some more traditional examples:
There’re heuristics for how to tell whether a person or organization is of bad character (even though these heuristics don’t tell how how to respond if a person is of bad character). Eg JK Rowling’s character Sirius’s claim that you can see the measure of a person by how they treat their house-elves (which has classical Christian antecedents, I’m just mentioning a contemporary phrasing).
There’re heuristics for how countries should be, e.g. “should have freedom of speech and press” or (longer ago) “should have a monarch who inherited legitimately.”
It would be too hard to try to equip humans and human groups for changing circumstances via only a “here’s what you do in situation X”. It’s somewhat easier to do it (and traditional ethical heuristics did do it) by a combination of “you can probably do well by [various what-to-do heuristics]” and “you can tell if you’re doing well by [various other checksum-type heuristics]. Ethics is help to let us design our way to better plans, not to only always give us those plans.
A key aspect of modern democracy with the rule of law is that companies can operate even if people believe they are acting with bad character. It’s not hard to convince a majority that Elon Musk and Sam Altman are people with bad character but that’s not sufficient to stopping them from building AGI.
As far as “should have freedom of speech and press” goes, both Republican and Democratic administrations over the last two decades did a lot to reduce those freedoms but the pushback comes mostly on partisan lines. They amount of people who take a principled stand on freedom of speech no matter whether it’s speech by friends or foes is small.
As far as “should have a monarch who inherited legitimately” goes, I think it worked for a long time as a Schelling point around with people could coordinate and not because most people found the concept of being ruled by a king that great. It was a Schelling point that allowed peaceful transition of power after a king died where otherwise there would have been more conflict about succession.
Eg JK Rowling’s character Sirius’s claim that you can see the measure of a person by how they treat their house-elves
While we are at general principles, citing JK Rowling in a discussion on ethics is probably generally a bad idea for politics is the mind killer reasons. I think the article is very interesting in terms of cultural norms.
It gets frequently cited to make a point that discussing politics is inherently bad, which isn’t something the article argues. On the other hand, the actual argument that if you use political examples it will make your audience focus on politics and make them less clear thinking when you could use non-political examples that don’t have this problem is seldomly appreciated, because people like using their political examples.
It’s a good point, re: some of the gap being that it’s hard to concretely visualize the world in which AGI isn’t built. And also about the “we” being part of the lack of concreteness.
I suspect there’re lots of kinds of ethical heuristics that’re supposed to interweave, and that some are supposed to be more like “checksums” (indicators everyone can use in an embodied way to see whether there’s a problem, even though they don’t say how to address it if there is a problem), and others are supposed to be more concrete.
For some more traditional examples:
There’re heuristics for how to tell whether a person or organization is of bad character (even though these heuristics don’t tell how how to respond if a person is of bad character). Eg JK Rowling’s character Sirius’s claim that you can see the measure of a person by how they treat their house-elves (which has classical Christian antecedents, I’m just mentioning a contemporary phrasing).
There’re heuristics for how countries should be, e.g. “should have freedom of speech and press” or (longer ago) “should have a monarch who inherited legitimately.”
It would be too hard to try to equip humans and human groups for changing circumstances via only a “here’s what you do in situation X”. It’s somewhat easier to do it (and traditional ethical heuristics did do it) by a combination of “you can probably do well by [various what-to-do heuristics]” and “you can tell if you’re doing well by [various other checksum-type heuristics]. Ethics is help to let us design our way to better plans, not to only always give us those plans.
A key aspect of modern democracy with the rule of law is that companies can operate even if people believe they are acting with bad character. It’s not hard to convince a majority that Elon Musk and Sam Altman are people with bad character but that’s not sufficient to stopping them from building AGI.
As far as “should have freedom of speech and press” goes, both Republican and Democratic administrations over the last two decades did a lot to reduce those freedoms but the pushback comes mostly on partisan lines. They amount of people who take a principled stand on freedom of speech no matter whether it’s speech by friends or foes is small.
As far as “should have a monarch who inherited legitimately” goes, I think it worked for a long time as a Schelling point around with people could coordinate and not because most people found the concept of being ruled by a king that great. It was a Schelling point that allowed peaceful transition of power after a king died where otherwise there would have been more conflict about succession.
While we are at general principles, citing JK Rowling in a discussion on ethics is probably generally a bad idea for politics is the mind killer reasons. I think the article is very interesting in terms of cultural norms.
It gets frequently cited to make a point that discussing politics is inherently bad, which isn’t something the article argues. On the other hand, the actual argument that if you use political examples it will make your audience focus on politics and make them less clear thinking when you could use non-political examples that don’t have this problem is seldomly appreciated, because people like using their political examples.