This is a better-argued version of something I feel like I’ve been circling for a while, thanks for writing it.
Something you didn’t suggest but I think might be a pitfall to avoid: I don’t think you can hill-climb on articulacy by getting (for example) Fable to explain things to Haiku. The ways in which a weak model misunderstands are (I claim) sufficiently different from the ways in which a low-context human misunderstands, that I don’t think weak LLMs are a good proxy for low-context humans
This is a better-argued version of something I feel like I’ve been circling for a while, thanks for writing it.
Something you didn’t suggest but I think might be a pitfall to avoid: I don’t think you can hill-climb on articulacy by getting (for example) Fable to explain things to Haiku. The ways in which a weak model misunderstands are (I claim) sufficiently different from the ways in which a low-context human misunderstands, that I don’t think weak LLMs are a good proxy for low-context humans