There are two different senses of partiality in preserving values: irreversible change, and loss of influence. Irreversible change benefits from reflection, deciding right now is predictably stupid. And quality of reflection doesn’t benefit from irreversible change you can point at right now, it’s prudent to prevent all change in order to carefully develop ability to choose it judiciously. On the other hand, lack of change is a competitive disadvantage, leads to loss of influence and inefficiency of reflection. But at least you don’t lose yourself in a predictably stupid way.
The synthesis is to address both, that’s the problem of creating a smarter-than-yourself assistant that’s aligned enough to give good advice on the process of reflection and help with not losing the future in the meantime. The assistant is change, but putting your values in control makes it reversible.
There are two different senses of partiality in preserving values: irreversible change, and loss of influence. Irreversible change benefits from reflection, deciding right now is predictably stupid. And quality of reflection doesn’t benefit from irreversible change you can point at right now, it’s prudent to prevent all change in order to carefully develop ability to choose it judiciously. On the other hand, lack of change is a competitive disadvantage, leads to loss of influence and inefficiency of reflection. But at least you don’t lose yourself in a predictably stupid way.
The synthesis is to address both, that’s the problem of creating a smarter-than-yourself assistant that’s aligned enough to give good advice on the process of reflection and help with not losing the future in the meantime. The assistant is change, but putting your values in control makes it reversible.