It seems as if you think of most people as cats. Does this mean that your AI safety work is largely motivated by ‘animal welfare’-like concerns, or do you mainly do it for the rest of the people who you don’t think of as cats?
Neither. I don’t want to die to AI, and I don’t want the universe to be optimized into a shape an AI likes rather than a shape I like (because I expect I would not like the AI’s preferred shape at all). Having lots of flourishing humans is one big part of a shape I like, but I’m not really doing it for their benefit.
A world I like would probably also include many cool flowers, but I’m not doing the work for the flowers’ benefit.
“Never use a person purely as means, but rather always also at least partially as ends in themselves” is the starting injunction from which to derive most of the other stuff.
Once you do that over and over, you’ll begin to notice regularities in the proof tactics and lemmas that come up, and think about how these logical structures would work if copied over and over...
...and another formulation that might lead to the “the same categorical imperative” IS SIMPLY just “do that which would be great if everyone did it” and then trying to unpack that logically in specific cases, noticing different roles, different promises, different duties...
...either way you eventually start seeing Natural Law, in the convergently (across situations) useful reasoning patterns that arise. You’re likely to notice that Natural Law is very big, and gets complex for N-person systems, and that you don’t understand it very well yet (and probably at its outer reaches it requires solving NP-hard optimization problems), but your life would go better if you did, and other people would be nicer to be around if they also understood it more.
You gain light context-sensitive attachments to some of it, and get the ability to warn people when you might have to predictably tit-for-tat them if they defect in predictable ways! You become a more morally mature person, who causes less accidental harm, and recognizes formal debts more reliably.
A coherently articulable conscience, based in an assumption of universal moral reasoning accessible at least in theory to all persons, leads to greater continence (a good word, used in many different ways by many different philosophers (I often just use the word to mean: less likely to metaphorically “pee” on stuff like an oblivious dog)) <3
But then yeah… you stop even “the cats” as cats.
You start seeing them as either willfully ignorant (incorrigible) monsters, or as childishly ignorant but essentially corrigible fools… or some variation on these themes, like maybe as developmentally disabled people who could hypothetically be fixed by raising their iq enough for them to learn to read and do basic arithmetic or whatever?
(For me, the Piraha are a deeply challenging test case for many moral theories, given their irremediable innumeracy and non-recursive grammar and so on… I currently suspect that a coherently good moral system would not allow them to vote or sign contracts, but I’m not sure.)
One of the great great great sadnesses of LW culture, from my perspective, is that the actual real original meaning of “incorrigible” and “corrigible” and so on (related to the degree of culpability for a harm caused by a person, based on mens rea and vincible ignorance and so on)… is overshadowed by Eliezer’s concept of “being a a super slavey slave, who is really mindless and literalistic and non-creative and passive, but is safe to use as a wish granting genie nonetheless (or somehow because of that)” that he happened to hang the world “corrigibility” on :-(
It seems as if you think of most people as cats. Does this mean that your AI safety work is largely motivated by ‘animal welfare’-like concerns, or do you mainly do it for the rest of the people who you don’t think of as cats?
Neither. I don’t want to die to AI, and I don’t want the universe to be optimized into a shape an AI likes rather than a shape I like (because I expect I would not like the AI’s preferred shape at all). Having lots of flourishing humans is one big part of a shape I like, but I’m not really doing it for their benefit.
A world I like would probably also include many cool flowers, but I’m not doing the work for the flowers’ benefit.
You… might need more Kant?
“Never use a person purely as means, but rather always also at least partially as ends in themselves” is the starting injunction from which to derive most of the other stuff.
Once you do that over and over, you’ll begin to notice regularities in the proof tactics and lemmas that come up, and think about how these logical structures would work if copied over and over...
...and another formulation that might lead to the “the same categorical imperative” IS SIMPLY just “do that which would be great if everyone did it” and then trying to unpack that logically in specific cases, noticing different roles, different promises, different duties...
...either way you eventually start seeing Natural Law, in the convergently (across situations) useful reasoning patterns that arise. You’re likely to notice that Natural Law is very big, and gets complex for N-person systems, and that you don’t understand it very well yet (and probably at its outer reaches it requires solving NP-hard optimization problems), but your life would go better if you did, and other people would be nicer to be around if they also understood it more.
You gain light context-sensitive attachments to some of it, and get the ability to warn people when you might have to predictably tit-for-tat them if they defect in predictable ways! You become a more morally mature person, who causes less accidental harm, and recognizes formal debts more reliably.
A coherently articulable conscience, based in an assumption of universal moral reasoning accessible at least in theory to all persons, leads to greater continence (a good word, used in many different ways by many different philosophers (I often just use the word to mean: less likely to metaphorically “pee” on stuff like an oblivious dog)) <3
But then yeah… you stop even “the cats” as cats.
You start seeing them as either willfully ignorant (incorrigible) monsters, or as childishly ignorant but essentially corrigible fools… or some variation on these themes, like maybe as developmentally disabled people who could hypothetically be fixed by raising their iq enough for them to learn to read and do basic arithmetic or whatever?
(For me, the Piraha are a deeply challenging test case for many moral theories, given their irremediable innumeracy and non-recursive grammar and so on… I currently suspect that a coherently good moral system would not allow them to vote or sign contracts, but I’m not sure.)
One of the great great great sadnesses of LW culture, from my perspective, is that the actual real original meaning of “incorrigible” and “corrigible” and so on (related to the degree of culpability for a harm caused by a person, based on mens rea and vincible ignorance and so on)… is overshadowed by Eliezer’s concept of “being a a super slavey slave, who is really mindless and literalistic and non-creative and passive, but is safe to use as a wish granting genie nonetheless (or somehow because of that)” that he happened to hang the world “corrigibility” on :-(