I appreciate your taking on the core issues and summarizing them. Bravo! Being paywalled, the material is not freely available in general.
You tried to enumerate all views but did not list quite the perspective I had: I think we can take solace from previous encounters between vast intelligence differences, for examples mushrooms versus people. This is a variant of “alignment is not hard” that is more like “sufficient alignment is not hard”. The “sufficient” part is very important. I completely agree with the IABIED perspective that AI alignment is hard, but all we need for survival is that the extreme worst anti-alignment does not come to pass.
So let’s look at mushrooms for a minute and come back to AI.
Mushrooms came first. Mushrooms, if we include fungi in general, go back over a billion years. Humanity goes back less than a million. Humans have dominated mushrooms in most categories and certainly in anything involving intelligence, but we’ve lived on earth together just fine and barely even notice each other are there.
I take some solice that this kind of thing happens all the time. A difference in capability does not generally lead to all-out war with no peace possible.
We know that this is true just from considering the evidence. Why it happens is a more speculative question, but here are two reasonsthat seem plausible to me:
1. To the extent that values are in the same plane as each other, it is often better to cooperate than to fight, even from a selfish point of view. Mushrooms and humans do both want food and sunlight. However, people are not wiping out mushrooms so that we can have all of those resources for ourselves. It’s the other way around! People generally prefer living in an environment with rich soils and thriving green plants, and those same environments are also good for mushrooms. We value a diverse ecosystem and feel like we are stronger when we can create it for ourselves. We even demarcate large natural reserves just for the principle of it.
2. To the extent our values are incomprehensible, we often have nothing to fight about. Humans care about finding a lover, building relationships of all kinds, about finding engagement in our community. When we make art, we value when people enjoy it and value getting credit for it. For things like this, mushrooms just don’t enter the competition at all. They don’t steal our lover and do not take credit for our songs. Humans therefore save our resources and put them into those things we do care about.
I picked mushrooms as an extreme example, but you can pick almost any two species. If you want to focus on intelligence, then it may be worth considering the way humans treat dogs and cats. We are not always kind to them but are really not in a hurry to wipe them all out.
I think your comment relates to the assumption that was mentioned is explicitly not covered in this post:
A misaligned ASI would cause human extinction and that would be undesirable. It’s possible that an ASI could be misaligned and have alien goals. Conversely, it’s also possible to create an ASI that would be aligned with human values (see the orthogonality thesis).
However, I do agree this view isn’t perfectly covered by the flowchart. I think it would be getting off at the “ASI is extremely powerful node”, but is more like “ASI isn’t sufficiently powerful that misalignment would lead to human extinction” which is more nuanced but directionally the same.
I appreciate your taking on the core issues and summarizing them. Bravo! Being paywalled, the material is not freely available in general.
You tried to enumerate all views but did not list quite the perspective I had: I think we can take solace from previous encounters between vast intelligence differences, for examples mushrooms versus people. This is a variant of “alignment is not hard” that is more like “sufficient alignment is not hard”. The “sufficient” part is very important. I completely agree with the IABIED perspective that AI alignment is hard, but all we need for survival is that the extreme worst anti-alignment does not come to pass.
So let’s look at mushrooms for a minute and come back to AI.
Mushrooms came first. Mushrooms, if we include fungi in general, go back over a billion years. Humanity goes back less than a million. Humans have dominated mushrooms in most categories and certainly in anything involving intelligence, but we’ve lived on earth together just fine and barely even notice each other are there.
I take some solice that this kind of thing happens all the time. A difference in capability does not generally lead to all-out war with no peace possible.
We know that this is true just from considering the evidence. Why it happens is a more speculative question, but here are two reasonsthat seem plausible to me:
1. To the extent that values are in the same plane as each other, it is often better to cooperate than to fight, even from a selfish point of view. Mushrooms and humans do both want food and sunlight. However, people are not wiping out mushrooms so that we can have all of those resources for ourselves. It’s the other way around! People generally prefer living in an environment with rich soils and thriving green plants, and those same environments are also good for mushrooms. We value a diverse ecosystem and feel like we are stronger when we can create it for ourselves. We even demarcate large natural reserves just for the principle of it.
2. To the extent our values are incomprehensible, we often have nothing to fight about. Humans care about finding a lover, building relationships of all kinds, about finding engagement in our community. When we make art, we value when people enjoy it and value getting credit for it. For things like this, mushrooms just don’t enter the competition at all. They don’t steal our lover and do not take credit for our songs. Humans therefore save our resources and put them into those things we do care about.
I picked mushrooms as an extreme example, but you can pick almost any two species. If you want to focus on intelligence, then it may be worth considering the way humans treat dogs and cats. We are not always kind to them but are really not in a hurry to wipe them all out.
I think your comment relates to the assumption that was mentioned is explicitly not covered in this post:
However, I do agree this view isn’t perfectly covered by the flowchart. I think it would be getting off at the “ASI is extremely powerful node”, but is more like “ASI isn’t sufficiently powerful that misalignment would lead to human extinction” which is more nuanced but directionally the same.