An ASI perfectly aligned to me must literally be a smarter version of myself. Anything less than that is a compromise between my values and the values of society. Such a compromise at its extreme fills me with dread. I would much rather live in a society of some discord between many individually aligned ASI’s, than build a benevolent god.
An ASI aligned to a group of people likely should dedicate sovereign slivers of compute (optimization domains) for each of those people, and those people could do well with managing their domain with their own ASIs aligned to each of them separately. Optimization doesn’t imply a uniform pureed soup, it’s also possible to optimize autonomy, coordination, and interaction, without mixing them up.
An ASI perfectly aligned to me must literally be a smarter version of myself.
Values judge what should be done, but also what you personally should be doing. An ASI value aligned to you will be doing the things that should be done (according to you, on reflection), but you wouldn’t necessarily endorse that you personally should be doing those things. Like, I want the world to be saved, but I don’t necessarily want to be in a position to need to try to save the world personally.
So an ASI perfectly aligned to you might help uplift you into a smarter version of yourself as one of its top priorities, and then go on to do various other things you’d approve of on reflection. But you wouldn’t necessarily endorse that it’s the smarter version of yourself that is doing those other things, you are merely endorsing that they get done.
I’m confused about that. I think you might be wrong, but I’ve heard this take before. If what you want is something that looks like a benevolent god, but one according to your own design, then that’s the “cosmopolitan empowerment by I just want cosmopolitanism” scenario. which I don’t trust, and so if I had the opportunity to design an AI, I would do my best to guarantee it’s cosmopolitanism-as-in-a-thing-others-actually-approve-of, for basically “values level LDT” reasons. see also interdimensional council of cosmopolitanisms
Anything less than that is a compromise between my values and the values of society.
I think there’s more leeway here. E.g. instead of a copy of you, a “friend” ASI.
I would much rather live in a society of some discord between many individually aligned ASI’s, than build a benevolent god
A benevolent god that understands your individual values and respects them seems pretty nice to me. Especially compared to a world of competing, individually aligned ASIs. (if your values are in the minority)
An ASI perfectly aligned to me must literally be a smarter version of myself. Anything less than that is a compromise between my values and the values of society. Such a compromise at its extreme fills me with dread. I would much rather live in a society of some discord between many individually aligned ASI’s, than build a benevolent god.
An ASI aligned to a group of people likely should dedicate sovereign slivers of compute (optimization domains) for each of those people, and those people could do well with managing their domain with their own ASIs aligned to each of them separately. Optimization doesn’t imply a uniform pureed soup, it’s also possible to optimize autonomy, coordination, and interaction, without mixing them up.
Values judge what should be done, but also what you personally should be doing. An ASI value aligned to you will be doing the things that should be done (according to you, on reflection), but you wouldn’t necessarily endorse that you personally should be doing those things. Like, I want the world to be saved, but I don’t necessarily want to be in a position to need to try to save the world personally.
So an ASI perfectly aligned to you might help uplift you into a smarter version of yourself as one of its top priorities, and then go on to do various other things you’d approve of on reflection. But you wouldn’t necessarily endorse that it’s the smarter version of yourself that is doing those other things, you are merely endorsing that they get done.
I’m confused about that. I think you might be wrong, but I’ve heard this take before. If what you want is something that looks like a benevolent god, but one according to your own design, then that’s the “cosmopolitan empowerment by I just want cosmopolitanism” scenario. which I don’t trust, and so if I had the opportunity to design an AI, I would do my best to guarantee it’s cosmopolitanism-as-in-a-thing-others-actually-approve-of, for basically “values level LDT” reasons. see also interdimensional council of cosmopolitanisms
I think there’s more leeway here. E.g. instead of a copy of you, a “friend” ASI.
A benevolent god that understands your individual values and respects them seems pretty nice to me. Especially compared to a world of competing, individually aligned ASIs. (if your values are in the minority)