I’d be curious to get a better sense of what sorts of RL environments you’re imagining. Math problems? Video game environments? Complicated multi-agent conversations? The near-term feasibility of AI generating novel RL environments seems like it varies dramatically depending on the answers.
There’s an efficient-market-hypothesis-style argument that everything’s priced in irrespective of the details, but I’m skeptical of that sort of argument in a context where the relevant players are bottlenecked on people and ability to test ideas.
I’m curious about this partly because DeepMind’s recently released Genie 3 (impressive flashy video, blog post) surprised me with how good it is, and seems like it plausibly hits the threshold at which high-quality video game RL environments can be generated at scale much more cheaply than an hour of developer time[1] (potentially triggering the kind of superexponential increase you talk about).
I’d be curious to get a better sense of what sorts of RL environments you’re imagining. Math problems? Video game environments? Complicated multi-agent conversations? The near-term feasibility of AI generating novel RL environments seems like it varies dramatically depending on the answers.
There’s an efficient-market-hypothesis-style argument that everything’s priced in irrespective of the details, but I’m skeptical of that sort of argument in a context where the relevant players are bottlenecked on people and ability to test ideas.
I’m curious about this partly because DeepMind’s recently released Genie 3 (impressive flashy video, blog post) surprised me with how good it is, and seems like it plausibly hits the threshold at which high-quality video game RL environments can be generated at scale much more cheaply than an hour of developer time[1] (potentially triggering the kind of superexponential increase you talk about).
Caveat: I’m not sure how expensive it is in compute; that could potentially offset the decreased cost in developer time.
Agentic software engineering mostly, I don’t think Genie matters.