I think this is directionally correct in terms of the size of the problem and the amount of funding necessary. I have concerns though on the structure. Broadly, my largest concerns are that a 1) hub-and-spoke funding structure (like we currently have) isn’t the best approach, long-term 2) the structure you’re proposing lends itself to siloed research, where every org is independently trying to hillclimb the safety proxy.
If we agree on the scale of the problem, wouldn’t we also agree that whatever small teams (METR/Apollo/Redwood size) would go further through open collaboration with academics, other labs, independent researchers, companies etc. I’ve written more here: Automating AI Safety Research Requires an Open Ecosystem, Not Bigger Grants.
Editing for clarity here: METRs agents are probably substantially behind the agent harnesses within the labs. How much are you discounting for the gap between METR’s agent and what’s running internally in the labs?