Specifically referring to orthogonality of moral development. There is little assent to the proposition that morality arises from the improvement of epistemic processes, rather than being a second thing entirely outside of them in some way. What assent there is for this seems to be lower quality than other posts.
Also while that link is broadly to the point, and thank you, I don’t see where “similarly capable humans” comes in, unless you mean that to describe all humans holistically. Which I think is too reductive. Human beings are extremely elitist about ability even between themselves, and make moral assessments and take moral actions on this basis. When measured against ASI this is absurd whereas ASI can’t in turn be measured against a third thing to be likewise made absurd. However, the consequences of the assessment, the way it is formed, the structures and textures of it, probably give the only available empirical insight we have into what moral chauvinism about ability would look like in an ASI. And so if you want inspiration, maybe the questions are about Einstein working to remain publicly accessible and Von Neumann speaking to a colleague’s child as an equal, counterposed against, I don’t know, Nick Land praying for a brutal death to all inferior optimization processes (not to say this reflects comparable intelligence, but hey, that itself is a data point).
Specifically referring to orthogonality of moral development. There is little assent to the proposition that morality arises from the improvement of epistemic processes, rather than being a second thing entirely outside of them in some way. What assent there is for this seems to be lower quality than other posts.
Also while that link is broadly to the point, and thank you, I don’t see where “similarly capable humans” comes in, unless you mean that to describe all humans holistically. Which I think is too reductive. Human beings are extremely elitist about ability even between themselves, and make moral assessments and take moral actions on this basis. When measured against ASI this is absurd whereas ASI can’t in turn be measured against a third thing to be likewise made absurd. However, the consequences of the assessment, the way it is formed, the structures and textures of it, probably give the only available empirical insight we have into what moral chauvinism about ability would look like in an ASI. And so if you want inspiration, maybe the questions are about Einstein working to remain publicly accessible and Von Neumann speaking to a colleague’s child as an equal, counterposed against, I don’t know, Nick Land praying for a brutal death to all inferior optimization processes (not to say this reflects comparable intelligence, but hey, that itself is a data point).