For me the math and CS research progress is a bigger update than the METR/Anthropic reporting… The ability to outperform large communities of human researchers at the task of designing and improving algorithms is a core part of an intelligence explosion, so I think that’s a really important indicator.
This was also a bigger update for me, and it seems like a similarly-sized one, so perhaps we don’t disagree that much. I’m curious, though, if you are differentiating between kinds of algorithm improvement/design. It seems like AI improving AI R&D could fall anywhere between “improving dataset quality” to “coming up with an AIXI-like algorithm from scratch.” My own sense is that while AI systems are getting better at a subset of algorithm improvement/design it does not yet extend to the latter kind of thing, nor does it seem obvious that they will get better at this latter kind of thing, and I think this plausibly matters for RSI. I’m not sure if/where we disagree though. Is your claim that AI systems are already outperforming humans at all types of algorithm design, or that they are not outperforming us at the “designing AIXI-like algorithms from scratch” type but will soon, or that they are not outperforming us at this and won’t but that this doesn’t bear on RSI? (Or another option I didn’t list)?
I’ve been tracking a distinction between well-specified versus non well-specified tasks for almost as long as I’ve been thinking seriously about AI, fwiw (since 2016 or so). I’ve been tracking it the entire time LLMs have existed in a pretty serious way. E.g. this is essentially what my PhD was about (starting in 2019) which I was interested in pursuing in part because I was noticing this delta between artificial and biological intelligence. I don’t think “well-specified tasks” is quite the right name of the category (I don’t think we know what the true name is), and I’ve used different words in the past, as well as noticed different aspects and refined more of what I mean over the years, but the core distinction has been live for me for a long time.