The impractical part about training for good behavior is that it’s a nested loop—every training example on how to find good maxima requires training a model that in turn needs its own training examples. So it’s destined to be behind the state of the art, probably using state of the art models to generate the copious required training data.
The question, I suppose, is whether this is still good enough to learn useful general lessons. And after thinking about it, I think the answer is that yes, it should be, especially for feed-forward architectures that look like modern machine learning, where you don’t expect qualitative changes in capability as you scale computational resources.
(Not 100% sure I understood your comment.) Training is one idea, but you could also just test out heuristics with this framework. For example, I think this scheme could be used to benchmark quantilization against a competing approach.
Pretty sure you understood it :) But yeah, not only would I like to be able to compare two things, I’d like to be able to find the optimum values of some continuous variables. Though I suppose it doesn’t matter as much if you’re trying to check / evaluate ideas that you arrived at by more abstract reasoning.
The impractical part about training for good behavior is that it’s a nested loop—every training example on how to find good maxima requires training a model that in turn needs its own training examples. So it’s destined to be behind the state of the art, probably using state of the art models to generate the copious required training data.
The question, I suppose, is whether this is still good enough to learn useful general lessons. And after thinking about it, I think the answer is that yes, it should be, especially for feed-forward architectures that look like modern machine learning, where you don’t expect qualitative changes in capability as you scale computational resources.
(Not 100% sure I understood your comment.) Training is one idea, but you could also just test out heuristics with this framework. For example, I think this scheme could be used to benchmark quantilization against a competing approach.
Pretty sure you understood it :) But yeah, not only would I like to be able to compare two things, I’d like to be able to find the optimum values of some continuous variables. Though I suppose it doesn’t matter as much if you’re trying to check / evaluate ideas that you arrived at by more abstract reasoning.