I think for me the critical thing within this picture is whether the current paradigm of RSI can expand the circle of capabilities it is capable of improving. If e.g superhuman maths unlocks new algorithm which includes more exploration and develops an AI with new non-human move 37 style ideas which… And this sort of thing happens even slowly, then you can still get to proper RSI.
It’s very unclear to me how to find out if this is likely to be the case or otherwise how to figure out if there is some sort of limiting barrier ahead of time.
I think for me the critical thing within this picture is whether the current paradigm of RSI can expand the circle of capabilities it is capable of improving. If e.g superhuman maths unlocks new algorithm which includes more exploration and develops an AI with new non-human move 37 style ideas which… And this sort of thing happens even slowly, then you can still get to proper RSI.
It’s very unclear to me how to find out if this is likely to be the case or otherwise how to figure out if there is some sort of limiting barrier ahead of time.