The scope of CEV is humanity: Not just the designer’s volition, and not the volition of non-human intelligences. Why?
If you exclude non-humans’ volition (except indirectly if humans care about it), then why not exclude the volition of all humans but the designer (except indirectly if the designer cares about it)?
If all humans’ volitions were identical or very similar, I could see an argument along the lines of Drescher or TDT. But they are not.
So, you could include the volition of all entities, but only the portion which overlaps with that of the designer. This would indeed consider humans much more than non-human intelligences. But that gets into dangerous territory, with educated Western liberals getting a higher weight (assuming that the usual crowd develops the AGI).
The scope of CEV is humanity: Not just the designer’s volition, and not the volition of non-human intelligences. Why?
If you exclude non-humans’ volition (except indirectly if humans care about it), then why not exclude the volition of all humans but the designer (except indirectly if the designer cares about it)?
If all humans’ volitions were identical or very similar, I could see an argument along the lines of Drescher or TDT. But they are not.
So, you could include the volition of all entities, but only the portion which overlaps with that of the designer. This would indeed consider humans much more than non-human intelligences. But that gets into dangerous territory, with educated Western liberals getting a higher weight (assuming that the usual crowd develops the AGI).