Thanks for your suggestions. I think having more people deeply engaged with alignment is good for our chances of getting it right.
I think this proposal falls into the category of goal crafting (a term proposed by Roko) - deciding what we want an AGI to do. Most alignment work addresses technical alignment—how we might get an AGI to reliably do anything. I think you imply the approach “just train it”; this might work for some types of AGI, and some types of training.
I think many humans trained in classical ethics are not actually ethical by their standards. It is one thing to understand and another to believe in an ethical system. My post “the partial.fqllacy of dumb superintelligence” is one of many treatments of that knowing vs caring distinction.
Thanks for the reply. To comment on your reply, let me be more precise in what I believe are the problems this approach could address.
a) Assuming that the AI is already loyal, or at least following our wishes: how can we make sure that it takes human values into account in its decisions such that it doesn’t mistakenly do something against these values?
I believe running every decision through a “Classical Virtue Ethics”-control, to be a way to do this.
As an analogy, we can train AI on grammar and rhetoric, by giving it a huge amount of literature to read, and it will be able to master the rules and applications of grammar and rhetoric. In the same way, it’s plausible that it would be possible to get it to master Classical humanist virtue ethics as well, seeing as that is just another art, and not different in principle.
So I propose that we could train it in virtue ethics in the same way it’s trained in grammar and rhetoric. If you call that “just train it”, and that is a dismissal of the approach, I’d appreciate some more in-depth engagement with the analogy to grammar and rhetoric.
b) Assuming that the AI isn’t “inwardly aligned”, but instead “ethically neutral”—at least before it’s trained, how can we increase the chance that it develops inward alignment with human values?
a )AI could seem to take on some of the values of the text it is trained on. If we train it on Classical virtue ethics, and seek to increase the value it gives to this input, we might increase the chances of it becoming “inwardly aligned”. Edit: it seems you are engaging with this type of idea in your “fallacy of dumb superintelligence”.
b) We could extrapolate from how humans become “aligned” with virtues. In classical virtue ethics one’s self-concept is central to this. And we get this alignment by emulating the role model, by seeking to adopt the role-model’s perspective. In the same manner, we could try to find a way to get the AI’s self-concept model to be tied to the classical virtues.
If we are lucky enough to find ourselves in one of in those situations you describe, where we have an AGI that wants to do what humanity wants (or would/should want) it to want to do, then to what degree is additional training actually required? I’m sure there are many possible such scenarios that vary widely, but my mental default assumption is that such a system would already be motivated to seek out answers to questions such as, “What is it the humans want me to want, and why?” which would naturally include studying… well, pretty much the entire moral philosophy literature. I wouldn’t put super high odds on it, though.
That said, one of my main concern about a classical virtue ethics training regimen specifically is that it doesn’t really give a clear answer about how to prioritize among virtues (or more broadly, things some subset of cultures says are virtues) that are in conflict, and real humans do in fact disagree and debate about this all the time.
This article explores the concept and potential application of bottom-up virtue ethics as an approach to instilling ethical behavior in artificial intelligence (AI) systems. We argue that by training machine learning models to emulate virtues such as honesty, justice, and compassion, we can cultivate positive traits and behaviors based on ideal human moral character. This bottom-up approach contrasts with traditional top-down programming of ethical rules, focusing instead on experiential learning. Although this approach presents its own challenges, it offers a promising avenue for the development of more ethically aligned AI systems.
Thanks for your suggestions. I think having more people deeply engaged with alignment is good for our chances of getting it right.
I think this proposal falls into the category of goal crafting (a term proposed by Roko) - deciding what we want an AGI to do. Most alignment work addresses technical alignment—how we might get an AGI to reliably do anything. I think you imply the approach “just train it”; this might work for some types of AGI, and some types of training.
I think many humans trained in classical ethics are not actually ethical by their standards. It is one thing to understand and another to believe in an ethical system. My post “the partial.fqllacy of dumb superintelligence” is one of many treatments of that knowing vs caring distinction.
Thanks for the reply. To comment on your reply, let me be more precise in what I believe are the problems this approach could address.
a) Assuming that the AI is already loyal, or at least following our wishes: how can we make sure that it takes human values into account in its decisions such that it doesn’t mistakenly do something against these values?
I believe running every decision through a “Classical Virtue Ethics”-control, to be a way to do this.
As an analogy, we can train AI on grammar and rhetoric, by giving it a huge amount of literature to read, and it will be able to master the rules and applications of grammar and rhetoric. In the same way, it’s plausible that it would be possible to get it to master Classical humanist virtue ethics as well, seeing as that is just another art, and not different in principle.
So I propose that we could train it in virtue ethics in the same way it’s trained in grammar and rhetoric. If you call that “just train it”, and that is a dismissal of the approach, I’d appreciate some more in-depth engagement with the analogy to grammar and rhetoric.
b) Assuming that the AI isn’t “inwardly aligned”, but instead “ethically neutral”—at least before it’s trained, how can we increase the chance that it develops inward alignment with human values?
a )AI could seem to take on some of the values of the text it is trained on. If we train it on Classical virtue ethics, and seek to increase the value it gives to this input, we might increase the chances of it becoming “inwardly aligned”. Edit: it seems you are engaging with this type of idea in your “fallacy of dumb superintelligence”.
b) We could extrapolate from how humans become “aligned” with virtues. In classical virtue ethics one’s self-concept is central to this. And we get this alignment by emulating the role model, by seeking to adopt the role-model’s perspective. In the same manner, we could try to find a way to get the AI’s self-concept model to be tied to the classical virtues.
If we are lucky enough to find ourselves in one of in those situations you describe, where we have an AGI that wants to do what humanity wants (or would/should want) it to want to do, then to what degree is additional training actually required? I’m sure there are many possible such scenarios that vary widely, but my mental default assumption is that such a system would already be motivated to seek out answers to questions such as, “What is it the humans want me to want, and why?” which would naturally include studying… well, pretty much the entire moral philosophy literature. I wouldn’t put super high odds on it, though.
That said, one of my main concern about a classical virtue ethics training regimen specifically is that it doesn’t really give a clear answer about how to prioritize among virtues (or more broadly, things some subset of cultures says are virtues) that are in conflict, and real humans do in fact disagree and debate about this all the time.
Note: I see this approach is also proposed by “AI Alignment Proposals”