What do you think of the point I brought up a few days ago about things like AlphaZero having infinitely large sample efficiency with respect to external data? If amplification improves datasets, it’s not crucial for learning itself to improve at all. From this point of view, the examples of extrapolation you gave are just salient hypotheses already in the model, a matter of training data. You are familiar with sines and spirals, so you can fit them. If amplification lets an AI play with math and plots, while following aesthetic sense already in the underlying human (language) model, these hypotheses are eventually going to be noticed.
I’ve recently started thinking that extrapolation and things like consequentialism/agency should be thought of as unsafe amplifications, those that generate data out of distribution without due scrutiny by existing sensibilities of the system (which isn’t possible for things too far out of distribution). So a possible AI safety maxim is to avoid extrapolation and optimization, the only optimization it’s allowed is that of learning models from data, and only safe datasets should be learned from (external datasets might become safer after augmentation by the system that adds in its attitude to what it sees). Instead it should work on expanding the distribution “at the boundary” with amplifications heavy on channeling existing attitudes and light on optimization pressure, such as short sessions of reflection.
What do you think of the point I brought up a few days ago about things like AlphaZero having infinitely large sample efficiency with respect to external data? If amplification improves datasets, it’s not crucial for learning itself to improve at all. From this point of view, the examples of extrapolation you gave are just salient hypotheses already in the model, a matter of training data. You are familiar with sines and spirals, so you can fit them. If amplification lets an AI play with math and plots, while following aesthetic sense already in the underlying human (language) model, these hypotheses are eventually going to be noticed.
I’ve recently started thinking that extrapolation and things like consequentialism/agency should be thought of as unsafe amplifications, those that generate data out of distribution without due scrutiny by existing sensibilities of the system (which isn’t possible for things too far out of distribution). So a possible AI safety maxim is to avoid extrapolation and optimization, the only optimization it’s allowed is that of learning models from data, and only safe datasets should be learned from (external datasets might become safer after augmentation by the system that adds in its attitude to what it sees). Instead it should work on expanding the distribution “at the boundary” with amplifications heavy on channeling existing attitudes and light on optimization pressure, such as short sessions of reflection.