IMO a core hard part of research automation seems to be adapting plans on the fly.
---
Research agents need to operate on long horizons under uncertainty.
Often I have a big experiment made of several stages, and the outcome of each stage is uncertain.
If stage one turns up a surprising result, running the rest as planned might be pointless — the right move is to reorient and update.
I want agents that can carry out the long-horizon plan, but also notice when something surprising happens and respond intelligently.
So: how should an agent update when it hits a surprise?
Option 1 — stop. Something unexpected happened, so halt and wait for instructions. This is safe — it caps the amount of tokens wasted on something that might be useless.
Option 2 — proceed, but update the plan intelligently. Harder. This requires commander’s intent: the research North Star needs to be specified up front, and the agent looks for a way to pivot that still makes progress toward it. Doing this well requires rich context about what you’re actually trying to do at a high level, let it pivot, and have it document its decisions along the way so you can ask about them later.
Option 2 is much closer to how I’d treat a human mentee. I suggest some experiments; they go off for a few days with low visibility; and when I check in they’ve often hit problems or done something different from the plan — but a good mentee will have made real progress anyway.
I’m not sure how to build this yet but it seems very valuable to have.
Speculative execution: To prevent serial bottlenecks (see next section), researchers may use two forms of speculative execution: starting lots of long experiments they’re not sure the project needs, and guessing results of experiments and feedback (see Tom Cunningham’s “Bottlenecks can be loosened with agents” section)
...
Once you can use agents to automate large parts of work it feels like you’ll now be bottlenecked on the non-automated parts. But in fact the non-automated parts can often be predicted, and this loosens the bottleneck.
Imagine every report has the following:
Agent’s best-guess about what comments you’d get from Beth, Hjalmar, Ajeya.
Agent’s best-guess about survey results, if you launched the survey.
Agent’s best-guess about benchmark results.
Agent’s best-guess about how this will be received on Twitter.
In addition you could click through to see why the agent guessed each. I feel these would meaningfully loosen bottlenecks, I could iterate until the information I received from the world (human feedback, data, surveys) was maximally informative, and only then send out for review.
IMO a core hard part of research automation seems to be adapting plans on the fly.
---
Research agents need to operate on long horizons under uncertainty.
Often I have a big experiment made of several stages, and the outcome of each stage is uncertain.
If stage one turns up a surprising result, running the rest as planned might be pointless — the right move is to reorient and update.
I want agents that can carry out the long-horizon plan, but also notice when something surprising happens and respond intelligently.
So: how should an agent update when it hits a surprise?
Option 1 — stop. Something unexpected happened, so halt and wait for instructions. This is safe — it caps the amount of tokens wasted on something that might be useless.
Option 2 — proceed, but update the plan intelligently. Harder. This requires commander’s intent: the research North Star needs to be specified up front, and the agent looks for a way to pivot that still makes progress toward it. Doing this well requires rich context about what you’re actually trying to do at a high level, let it pivot, and have it document its decisions along the way so you can ask about them later.
Option 2 is much closer to how I’d treat a human mentee. I suggest some experiments; they go off for a few days with low visibility; and when I check in they’ve often hit problems or done something different from the plan — but a good mentee will have made real progress anyway.
I’m not sure how to build this yet but it seems very valuable to have.
I pattern matched your option 2 to speculative execution, with agents you can branch off whenever the agent reach an uncertainty on the research goal.
https://metr.org/notes/2026-03-19-org-uplift-game/