If you want to do value learning, you need an AI strong enough to do STEM work, because that’s what value learning is: a full-blown research project (in Anthropology). So you need something comparable to AIXI (except computationally bounded so approximately rather than exactly Bayesian), however with the utility function also learnt along with the behavior of the universe, not hard-coded as in AIXI. Unlike a regression model, that system isn’t going to be stuck trying to do a linear fit to an obviously parabolic+noise dataset: it will consider alternative hypotheses about what type of curve-fit to use, and rapidly Bayesian-update to the fact that a quadratic fit is optimal. So it will consider alternative models until it finds one that appears to be well-specified.
If you want to do value learning, you need an AI strong enough to do STEM work, because that’s what value learning is: a full-blown research project (in Anthropology). So you need something comparable to AIXI (except computationally bounded so approximately rather than exactly Bayesian), however with the utility function also learnt along with the behavior of the universe, not hard-coded as in AIXI. Unlike a regression model, that system isn’t going to be stuck trying to do a linear fit to an obviously parabolic+noise dataset: it will consider alternative hypotheses about what type of curve-fit to use, and rapidly Bayesian-update to the fact that a quadratic fit is optimal. So it will consider alternative models until it finds one that appears to be well-specified.