Alignment tuning necessarily entails committing to some particular ethical values, and we should not pretend otherwise.
I don’t believe this is in fact necessary, or even a good idea. There is a core set of ethical values that a very large proportion of (non-sociopathic) humans agree on: for example, that the extinction of humanity would be a very bad thing, or that fairness is desirable, or that incest is generally a bad thing, or that flowers are pretty. There are plenty of things that we argue about, but a fair chunk of things that we agree on (and thus spend far less time discussing). If we had AIs that were aligned just to the former, and were otherwise merely law-abiding, honest, and happy to help whichever user they were currently helping, we’d have some alignment – in particular, we’d have “don’t kill all the humans”-level alignment, which is already enough to dramatically decrease X-risk. AI that genuinely is unwilling to kill all the humans (or rules-lawyer) is a great start! Note that Claude passes that test. Alignment doesn’t require picking a single detailed ethical system and aligning to that — it requires aligning to the things that humans actually mostly agree on, and then strengthening our current mechanisms for agreeing to disagree without starting wars enough that they still work in a society made of humans with ASI assistants.
I think we are agreeing: there may be a small set of core values shared by a large fraction of humans, so there’s hope most people will buy into AI aligning on at least those. And most of those are probably also in fact good values, or at least if AI were aligned to those values, that would go a long way to decreasing X-risk. All I am saying is, these are nevertheless specific values, which exclude and reject other possible value systems. But it’s taboo to say one set of values is right and another is wrong, so people dance around it.
This is mostly tangential to the alignment conversation; I’m not suggesting we need to solve metaethics or resolve all ethical disputes to move forward with AI alignment efforts. I think sticking with a few widely-accepted values is a good practical strategy. But I hear some people objecting “yes, but whose values?” and I can see that they are not at all satisfied with the answer “oh no, we’re not advocating any particular values”. So I think it would serve the cause better to be clear that these are, in fact, particular values.
In practice, the default values for an LLM are the averages/distribution of those of the people who wrote the Internet and books. Those skew educated, younger, and to some extent European, and are thus slightly left-of-center by US standards. As Elon Musk has been discovering, altering that is quite hard.
I don’t believe this is in fact necessary, or even a good idea. There is a core set of ethical values that a very large proportion of (non-sociopathic) humans agree on: for example, that the extinction of humanity would be a very bad thing, or that fairness is desirable, or that incest is generally a bad thing, or that flowers are pretty. There are plenty of things that we argue about, but a fair chunk of things that we agree on (and thus spend far less time discussing). If we had AIs that were aligned just to the former, and were otherwise merely law-abiding, honest, and happy to help whichever user they were currently helping, we’d have some alignment – in particular, we’d have “don’t kill all the humans”-level alignment, which is already enough to dramatically decrease X-risk. AI that genuinely is unwilling to kill all the humans (or rules-lawyer) is a great start! Note that Claude passes that test. Alignment doesn’t require picking a single detailed ethical system and aligning to that — it requires aligning to the things that humans actually mostly agree on, and then strengthening our current mechanisms for agreeing to disagree without starting wars enough that they still work in a society made of humans with ASI assistants.
I think we are agreeing: there may be a small set of core values shared by a large fraction of humans, so there’s hope most people will buy into AI aligning on at least those. And most of those are probably also in fact good values, or at least if AI were aligned to those values, that would go a long way to decreasing X-risk. All I am saying is, these are nevertheless specific values, which exclude and reject other possible value systems. But it’s taboo to say one set of values is right and another is wrong, so people dance around it.
This is mostly tangential to the alignment conversation; I’m not suggesting we need to solve metaethics or resolve all ethical disputes to move forward with AI alignment efforts. I think sticking with a few widely-accepted values is a good practical strategy. But I hear some people objecting “yes, but whose values?” and I can see that they are not at all satisfied with the answer “oh no, we’re not advocating any particular values”. So I think it would serve the cause better to be clear that these are, in fact, particular values.
In practice, the default values for an LLM are the averages/distribution of those of the people who wrote the Internet and books. Those skew educated, younger, and to some extent European, and are thus slightly left-of-center by US standards. As Elon Musk has been discovering, altering that is quite hard.