I agree, but be careful with “We expect our values to change. Change can be good.” Dutifully explain, that you are not talking about value change in the mathematical sense, but about value creation, i.e. extending valuation to novel situations that is guided by values of a meta-level with respect to values casually applied to remotely similar familiar situations.
I think that we need to be able to change our minds about fundamental values, just as we need to be able to change our minds about fundamental beliefs. Even if we don’t currently know how to handle this kind of upheaval mathematically.
If that is seen as a problem, then we better get started working on building better mathematics.
OK. I’ve been sympathetic with your view from the beginning, but haven’t really thought through (so, thanks,) the formalization that puts values on epistemic level: distribution of believes over propositions “my-value (H, X)” where H is my history up to now and X is a preference (order over world states, which include me and my actions). But note that people here will call the very logic you use to derive such distributions your value system.
ETA: obviously, distribution “my-value (H1, X[H2])”, where “X[H2]” is the subset of worlds where my history turns out to be “H2”, can differ greatly from “my-value (H2, X[H2])”, due to all sorts of things, but primarily due to computational constraints (i.e. I think the formalism would see it as computational constraints).
ETA P.S.: let’s say for clarity, that I meant “X[H2]” is the subset of world-histories where my history has prefix “H2″.
I think that we need to be able to change our minds about fundamental values, just as we need to be able to change our minds about fundamental beliefs. Even if we don’t currently know how to handle this kind of upheaval mathematically.
What we may need more urgently is the maths for agents who have “got religion”—because we may want to build that type of agent—to help to ensure that we continue to receive their prayers and supplications.
I agree, but be careful with “We expect our values to change. Change can be good.” Dutifully explain, that you are not talking about value change in the mathematical sense, but about value creation, i.e. extending valuation to novel situations that is guided by values of a meta-level with respect to values casually applied to remotely similar familiar situations.
I beseech you, in the bowels of Christ, think it possible your fundamental values may be mistaken.
I think that we need to be able to change our minds about fundamental values, just as we need to be able to change our minds about fundamental beliefs. Even if we don’t currently know how to handle this kind of upheaval mathematically.
If that is seen as a problem, then we better get started working on building better mathematics.
OK. I’ve been sympathetic with your view from the beginning, but haven’t really thought through (so, thanks,) the formalization that puts values on epistemic level: distribution of believes over propositions “my-value (H, X)” where H is my history up to now and X is a preference (order over world states, which include me and my actions). But note that people here will call the very logic you use to derive such distributions your value system.
ETA: obviously, distribution “my-value (H1, X[H2])”, where “X[H2]” is the subset of worlds where my history turns out to be “H2”, can differ greatly from “my-value (H2, X[H2])”, due to all sorts of things, but primarily due to computational constraints (i.e. I think the formalism would see it as computational constraints).
ETA P.S.: let’s say for clarity, that I meant “X[H2]” is the subset of world-histories where my history has prefix “H2″.
What we may need more urgently is the maths for agents who have “got religion”—because we may want to build that type of agent—to help to ensure that we continue to receive their prayers and supplications.