The inference/compute tradeoff if attempted to apply to people − 100 mathematicians with less skill (like less model training) cannot achieve what 1 can achieve even with a large amount of time. Einstein vs all the other worlds physicists at time, and Kasparov vs the world at chess etc. Specific concepts such as working memory are relevant for people, perhaps for AI as well.
OpenAI appears to be really struggling with inference costs—according to this article they should be continually retraining GPT 4 to reduce inference costs for the deployed model.
“for example, writing a program that passes some automatic tests”—Well yes—I and many people would probably want the option to pay for that - i.e. we specifically buy more inference to get improved performance.
“For example, AI companies might be able to use augmented models to speed up their own AI research.”—YES good point, I expect by the end of this article that point would have occurred to most of the audience on this site.
Interesting ideas. Some comments
The inference/compute tradeoff if attempted to apply to people − 100 mathematicians with less skill (like less model training) cannot achieve what 1 can achieve even with a large amount of time. Einstein vs all the other worlds physicists at time, and Kasparov vs the world at chess etc. Specific concepts such as working memory are relevant for people, perhaps for AI as well.
OpenAI appears to be really struggling with inference costs—according to this article they should be continually retraining GPT 4 to reduce inference costs for the deployed model.
“for example, writing a program that passes some automatic tests”—Well yes—I and many people would probably want the option to pay for that - i.e. we specifically buy more inference to get improved performance.
“For example, AI companies might be able to use augmented models to speed up their own AI research.”—YES good point, I expect by the end of this article that point would have occurred to most of the audience on this site.