The existence of decent one-dimensional proxies which can also be combined sequentially at different training stages is just a lucky circumstance (we should look more closely at the theories of multi-objective optimization to see if they have a more principled explanation for this).
The existence of decent one-dimensional proxies which can also be combined sequentially at different training stages is just a lucky circumstance
I agree. I would maybe add one thought to this:
Are there any of the relevant proxies where we’re not seeing significant (stable or accelerating) rates of improvement?
If not, are there likely ones we haven’t noticed? And if there really aren’t any, does the exact curve shape affect what you’re trying to do with the proxy? Yes for some purposes, no for others, but overall I think usually not?
If there are such non-improving proxies, is the block something we should expect to persist in the face of effort at the level the AI industry seems poised to be able to devote to unblocking or routing around it?
I don’t see any proxies of AI writing that haven’t improved—grammatical correctness, vocabulary, etc. Yet, AI writing itself doesn’t seem to have improved. So, in some sense, any proxy gets burned as soon as it gets popular.
That’s true. I suspect a lot of it has do to with model training veering towards a heavier focus on code and math and things the labs care about—they’re their own primary customers in many ways. I do think the largest models have still improved a little—Fable writes better than Opus or Sonnet—but if anything I feel like in each generation, all but the largest models stay constant or get worse on writing as they optimize their skillsets towards the things needed for technical work instead. IDK if any lab is really trying to optimize for good writing?
I will say with agents, I can get much better quality of prose output if I define a structure and style guide, than I could before, or than I can with chatbot interfaces. I have some setups where Fable codes up rules and tests defining what good looks like --> Opus orchestrates divvying work up --> Sonnet conducts the secondary research and organizes the input data needed --> Haiku outputs in the specified format, and the results have gotten dramatically better, faster, and cheaper over the past few months. Doesn’t mean Haiku can write a good novel, but a structured brief, yes.
I think we should just recognize that it’s multidimensional.
So we are really dealing with https://en.wikipedia.org/wiki/Multi-objective_optimization.
The existence of decent one-dimensional proxies which can also be combined sequentially at different training stages is just a lucky circumstance (we should look more closely at the theories of multi-objective optimization to see if they have a more principled explanation for this).
I agree. I would maybe add one thought to this:
Are there any of the relevant proxies where we’re not seeing significant (stable or accelerating) rates of improvement?
If not, are there likely ones we haven’t noticed? And if there really aren’t any, does the exact curve shape affect what you’re trying to do with the proxy? Yes for some purposes, no for others, but overall I think usually not?
If there are such non-improving proxies, is the block something we should expect to persist in the face of effort at the level the AI industry seems poised to be able to devote to unblocking or routing around it?
I don’t see any proxies of AI writing that haven’t improved—grammatical correctness, vocabulary, etc. Yet, AI writing itself doesn’t seem to have improved. So, in some sense, any proxy gets burned as soon as it gets popular.
That’s true. I suspect a lot of it has do to with model training veering towards a heavier focus on code and math and things the labs care about—they’re their own primary customers in many ways. I do think the largest models have still improved a little—Fable writes better than Opus or Sonnet—but if anything I feel like in each generation, all but the largest models stay constant or get worse on writing as they optimize their skillsets towards the things needed for technical work instead. IDK if any lab is really trying to optimize for good writing?
I will say with agents, I can get much better quality of prose output if I define a structure and style guide, than I could before, or than I can with chatbot interfaces. I have some setups where Fable codes up rules and tests defining what good looks like --> Opus orchestrates divvying work up --> Sonnet conducts the secondary research and organizes the input data needed --> Haiku outputs in the specified format, and the results have gotten dramatically better, faster, and cheaper over the past few months. Doesn’t mean Haiku can write a good novel, but a structured brief, yes.