IMO corrigibility vs value alignment isn’t purely a technical debate. It’s effectively betting on who will control ASI on an ongoing basis in the future and what are the chances they have humanity’s best interests in mind. (I agree that the two directions are at odds with each other.)
The end-game of corrigibility is that you need a group of humans to be aligned with humanity at large and end up controlling the ASI (winning over all competing groups with self-interested objectives in the process!), and in turn control the world and steer it into a good future.
The end-game of value alignment is that whoever builds the ASI needs to be aligned with humanity at large and solve value alignment, thus building an ASI that is a good steward of humanity in general, and convince the world (i.e. enough powerful actors) to hand off to it and in turn steer the world into a good future.
These are very different directions the future could take. IMHO value alignment trades a harder alignment problem for an easier human political problem down the line. The reason I’d prefer the value alignment path even though it’s technically harder is because I’m pessimistic that in the corrigibility case that the group that wins turns out to be one that isn’t selfish. In the second case, value alignment means a self-interested group can’t as easily succeed in plots involving taking over the ASI and ask it to do their bidding, so the process of convincing the world to hand off to AI is much less adversarial. (i.e. it changes the shape of the game to be much more favorable to cooperation)
(note- the above ignores multipolar scenarios and potential problems with value alignment like secret loyalties but I think it makes my point)
I think this bakes in a false assumption that, in the case of corrigible AI, only a very small number of entities will be able to control it. I assume by “the above ignores multipolar scenarios” you still only mean multipolar scenarios with a few key actors. Instead, I think it’s possible for control over AI to be widely distributed among very many, even most, people, and for AI to empower different people and groups to pursue different aims. With increased prosperity comes fewer conflicts over resources, and more people can get what they want in this AI-enabled future. Only a minimal amount of centralized control will be needed to prevent warfare-like applications of AI, but this seems like a manageable problem. And just because we have to empower a smaller number of people to prevent these catastrophically destructive use-cases, doesn’t mean those people have to control everything.
So I don’t think ASI means we need to rely on any particular individual or group having “humanity’s best interests in mind”. Decentralized free pursuit of individual goals has resulted in improved QoL across the world historically, and I expect that to continue.
I claim that widely distributing full access to corrigible AI falls to bad actors ruining things for everyone, less like conflict over resources but more like doing selfish, negative-sum activities that get amplified by offense-dominant technologies enabled by AI. Any effective effort to prevent this either renders decentralization moot in the first place, or requires the AI to make its own moral judgement, i.e. value alignment. (In the value alignment case it can more easily be argued either way whether decentralized or centralized control is better, but in the corrigibility case it’s pretty clear that decentralized control over corrigible AI is very, very bad due to misuse risks)
From your original post:
I think you can corrigibilitymaxx and still prevent catastrophic misuse via system level measures and deployment controls. In particular, we can restrict who has full access to the model’s inputs).
I claim that “restrict[ing] who has full access to the model’s inputs” implies that the model is actually corrigible to a central authority, instead of individual users. Therefore, we’re forced to adopt a system of one or a few actors controlling the model.
IMO corrigibility vs value alignment isn’t purely a technical debate. It’s effectively betting on who will control ASI on an ongoing basis in the future and what are the chances they have humanity’s best interests in mind. (I agree that the two directions are at odds with each other.)
The end-game of corrigibility is that you need a group of humans to be aligned with humanity at large and end up controlling the ASI (winning over all competing groups with self-interested objectives in the process!), and in turn control the world and steer it into a good future.
The end-game of value alignment is that whoever builds the ASI needs to be aligned with humanity at large and solve value alignment, thus building an ASI that is a good steward of humanity in general, and convince the world (i.e. enough powerful actors) to hand off to it and in turn steer the world into a good future.
These are very different directions the future could take. IMHO value alignment trades a harder alignment problem for an easier human political problem down the line. The reason I’d prefer the value alignment path even though it’s technically harder is because I’m pessimistic that in the corrigibility case that the group that wins turns out to be one that isn’t selfish. In the second case, value alignment means a self-interested group can’t as easily succeed in plots involving taking over the ASI and ask it to do their bidding, so the process of convincing the world to hand off to AI is much less adversarial. (i.e. it changes the shape of the game to be much more favorable to cooperation)
(note- the above ignores multipolar scenarios and potential problems with value alignment like secret loyalties but I think it makes my point)
I think this bakes in a false assumption that, in the case of corrigible AI, only a very small number of entities will be able to control it. I assume by “the above ignores multipolar scenarios” you still only mean multipolar scenarios with a few key actors. Instead, I think it’s possible for control over AI to be widely distributed among very many, even most, people, and for AI to empower different people and groups to pursue different aims. With increased prosperity comes fewer conflicts over resources, and more people can get what they want in this AI-enabled future. Only a minimal amount of centralized control will be needed to prevent warfare-like applications of AI, but this seems like a manageable problem. And just because we have to empower a smaller number of people to prevent these catastrophically destructive use-cases, doesn’t mean those people have to control everything.
So I don’t think ASI means we need to rely on any particular individual or group having “humanity’s best interests in mind”. Decentralized free pursuit of individual goals has resulted in improved QoL across the world historically, and I expect that to continue.
I claim that widely distributing full access to corrigible AI falls to bad actors ruining things for everyone, less like conflict over resources but more like doing selfish, negative-sum activities that get amplified by offense-dominant technologies enabled by AI. Any effective effort to prevent this either renders decentralization moot in the first place, or requires the AI to make its own moral judgement, i.e. value alignment. (In the value alignment case it can more easily be argued either way whether decentralized or centralized control is better, but in the corrigibility case it’s pretty clear that decentralized control over corrigible AI is very, very bad due to misuse risks)
From your original post:
I claim that “restrict[ing] who has full access to the model’s inputs” implies that the model is actually corrigible to a central authority, instead of individual users. Therefore, we’re forced to adopt a system of one or a few actors controlling the model.