I’m not sure if it’s what your thinking of, but I’m thinking of “What action is best according to these values” == “maximize reward”. One alternative that’s worth investigating more (IMO) is imposing hard constraints.
For instance, you could have an RL agent taking actions in $(a_1, a_2) \in \mathbb{R}^2$, and impose the constraint that $a_1 + a_2 < 3$ by projection.
“But I’m not sure what the alternative would be.”
I’m not sure if it’s what your thinking of, but I’m thinking of “What action is best according to these values” == “maximize reward”. One alternative that’s worth investigating more (IMO) is imposing hard constraints.
For instance, you could have an RL agent taking actions in $(a_1, a_2) \in \mathbb{R}^2$, and impose the constraint that $a_1 + a_2 < 3$ by projection.
A recent near-term safety paper takes this approach: https://arxiv.org/abs/1801.08757