(This is object-level about a question different from what I discuss in the post; I’ll reply only to the two specific obstacles raised, but generally, do not want to continue this discussion here, sorry.)
one can create a narrow recursive-self-improving AI that can code current AI and do coding better than people
My mainline expectation is that full-on RSI happens after everyone’s dead. One does not need to understand cognition on the level of being able to express it in code in order to find it via gradient descent, and improvements to the NN architecture / to the search process do not depend on understanding of how minds work.
AGI can be created with current methods, but it will be so costly from energy consumption point of view that real people will be more efficient
Once you have superintelligence, it gets compute (e.g., by stealing it or buying it with stolen money) regardless of whether it’s doing tasks that are sufficiently economically valuable to you to justify the compute cost.
Due to algorithmic and other improvements, the cost of inference at every level of intelligence goes down with time; current frontier models, even without being generally intelligent, are normally totally worth the cost the amount one would be willing to spend on general intelligence is higher than what we’re spending on the current models.
(This is object-level about a question different from what I discuss in the post; I’ll reply only to the two specific obstacles raised, but generally, do not want to continue this discussion here, sorry.)
My mainline expectation is that full-on RSI happens after everyone’s dead. One does not need to understand cognition on the level of being able to express it in code in order to find it via gradient descent, and improvements to the NN architecture / to the search process do not depend on understanding of how minds work.
Once you have superintelligence, it gets compute (e.g., by stealing it or buying it with stolen money) regardless of whether it’s doing tasks that are sufficiently economically valuable to you to justify the compute cost.
Due to algorithmic and other improvements, the cost of inference at every level of intelligence goes down with time; current frontier models, even without being generally intelligent, are normally totally worth the cost the amount one would be willing to spend on general intelligence is higher than what we’re spending on the current models.
Generally, beware the multiple stage fallacy.