This is great, thanks so much for pulling this together (and for linking to our Gato explainer!)
It just so happens I’m working with a group of people through the Cambridge EA Technical AI alignment curriculum, and this idea of IDA is what week 5 is all about—lots of further reading for those who want.
One prompt in the weekly curriculum asks whether there are any tasks that cannot easily be broken down in the way described above, and therefore might not be useful for IDA. One thing I can thing of offhand is large leaps in scientific understanding. For example, if you took 20 physicists and gave them the problems of the day, it’s not clear that they ever would have come up with Einstein’s theory of relativity. Given that problem, I wonder what the implications are for trying to use IDA to create AGI—does this mean there are certain types of tasks that a IDA-based AGI will not be so good at?
Thanks for reaching out! At the moment it’s only here and on the EA forum; we may be publishing it elsewhere but that’s still in the works. If you want to PM me with your request I’ll see what I can do!