In the field of AI we talk about capabilities vs alignment. I think it is relevant outside of the field of AI though.
I’m thinking back to something I read in Cal Newport’s book Digital Minimalism. He talked about how the Amish aren’t actually anti-technology. They are happy to adopt technology. They just want to make sure that the technology actually does more good than harm before they adopt it.
And thy have a neat process for this. From what I remember, they first start by researching it. Then have small groups of people experiment with it for some amount of time. Then larger groups. Something like that.
On the other hand, the impression I get is that we (strongly tend towards) assume that an increase in capabilities is automatically a good thing. For example, if there is some advancement made in the field of physics where we better understand subatomic particles the thought process is that this is exciting because down the line that theoretical understanding will lead to cool new technologies that improve our lives. This strikes me as being similar to the planning fallacy though: focusing on the “happy path” where things go the way you want them to go, and failing to think about the scenario where unexpected, bad things happen. Like next-gen nuclear weapons.
Speaking very generally, to me, it is very frequently not obvious whether capabilities improvements are actually aligned with our values and I’m not particularly excited when I hear about advancements in any given field.
From my perspective, part of the issue of this post is I notice a type error in the post when it talks about capabilities improvements being aligned with our values.
The question is, which values, and whose values are we talking about? Admittedly this is a common issue with morality, but in this case of capabilities research, this matters as our aligning it to our values is too vague to make sense. We need to go deeper and more concrete here so that we talk about specifically what we want our capabilities research is aligned to what values.
Yeah, I do agree that “values” is ambiguous. However, I think that is ok for the point that I’m making about capabilities vs alignment. Even though people don’t fully agree on values, paying more attention to alignment and being more careful about capabilities advancements still seems wise.
Capabilities vs alignment outside of AI
In the field of AI we talk about capabilities vs alignment. I think it is relevant outside of the field of AI though.
I’m thinking back to something I read in Cal Newport’s book Digital Minimalism. He talked about how the Amish aren’t actually anti-technology. They are happy to adopt technology. They just want to make sure that the technology actually does more good than harm before they adopt it.
And thy have a neat process for this. From what I remember, they first start by researching it. Then have small groups of people experiment with it for some amount of time. Then larger groups. Something like that.
On the other hand, the impression I get is that we (strongly tend towards) assume that an increase in capabilities is automatically a good thing. For example, if there is some advancement made in the field of physics where we better understand subatomic particles the thought process is that this is exciting because down the line that theoretical understanding will lead to cool new technologies that improve our lives. This strikes me as being similar to the planning fallacy though: focusing on the “happy path” where things go the way you want them to go, and failing to think about the scenario where unexpected, bad things happen. Like next-gen nuclear weapons.
Speaking very generally, to me, it is very frequently not obvious whether capabilities improvements are actually aligned with our values and I’m not particularly excited when I hear about advancements in any given field.
From my perspective, part of the issue of this post is I notice a type error in the post when it talks about capabilities improvements being aligned with our values.
The question is, which values, and whose values are we talking about? Admittedly this is a common issue with morality, but in this case of capabilities research, this matters as our aligning it to our values is too vague to make sense. We need to go deeper and more concrete here so that we talk about specifically what we want our capabilities research is aligned to what values.
Yeah, I do agree that “values” is ambiguous. However, I think that is ok for the point that I’m making about capabilities vs alignment. Even though people don’t fully agree on values, paying more attention to alignment and being more careful about capabilities advancements still seems wise.