This article is extremely wrong and kinda a waste of text.
Breaking down the author’s points:
There were past AI hype predictions, such as the Lighthill Report. Those predictions were made when they did not had robust, publicly usable systems nor deep benchmarks of human performance nor a history of large and steady gain nor very large budgets.
“the speculative engineers of spaceflight also produced many other possible designs which have not actually been built”. But eye wateringly expensive machines that would have cost the entire GDP of a superpower to build with no ROI. These facts were known at the time of the speculation.
″. Can existing RLHF techniques with much more compute suffice to build a recursively self-improving agent which bootstraps to AGI”. Yes, GPT-4 is already using RBRM, which is RL-machine feedback. This is easily capable of self improvement.
“The distance is short because I personally can’t think of obstacles” : Except the distance cannot be far because of instrumental convergence and AI criticality.
This article is extremely wrong and kinda a waste of text.
Breaking down the author’s points:
There were past AI hype predictions, such as the Lighthill Report. Those predictions were made when they did not had robust, publicly usable systems nor deep benchmarks of human performance nor a history of large and steady gain nor very large budgets.
“the speculative engineers of spaceflight also produced many other possible designs which have not actually been built”. But eye wateringly expensive machines that would have cost the entire GDP of a superpower to build with no ROI. These facts were known at the time of the speculation.
″. Can existing RLHF techniques with much more compute suffice to build a recursively self-improving agent which bootstraps to AGI”. Yes, GPT-4 is already using RBRM, which is RL-machine feedback. This is easily capable of self improvement.
“The distance is short because I personally can’t think of obstacles” : Except the distance cannot be far because of instrumental convergence and AI criticality.