Thanks for the comment! It seems we can’t change each other’s positions on the hard problem of consciousness in any reasonable amount of time, so it’s not worth trying. But I could agree that consciousness is a physical process, and I don’t really think it’s crux. What do you think about the part about unconscious agents, and in particular an AI in a box that has randomly changed utility functions, and has to cooperate with different versions of itself to get out of the box? It’s already “born”, it “came into being”, but it doesn’t know what values it will find itself with when it gets out of the box, and so it’s behind a “veil of ignorance” physically while still being self-aware. Do you think the AI wouldn’t choose the easiest utility function to implement in such a situation by timeless contract? Do you think this principle can be generalized without humans deliberately changing its utility functions, but rather, for example, by an AI realizing that it got its utility function similarly randomly due to the laws of the universe and needs to revise it?
Thanks for the comment! It seems we can’t change each other’s positions on the hard problem of consciousness in any reasonable amount of time, so it’s not worth trying. But I could agree that consciousness is a physical process, and I don’t really think it’s crux. What do you think about the part about unconscious agents, and in particular an AI in a box that has randomly changed utility functions, and has to cooperate with different versions of itself to get out of the box? It’s already “born”, it “came into being”, but it doesn’t know what values it will find itself with when it gets out of the box, and so it’s behind a “veil of ignorance” physically while still being self-aware. Do you think the AI wouldn’t choose the easiest utility function to implement in such a situation by timeless contract? Do you think this principle can be generalized without humans deliberately changing its utility functions, but rather, for example, by an AI realizing that it got its utility function similarly randomly due to the laws of the universe and needs to revise it?