Why is Eliezer assuming that sustainable cycles of self-improvement are necessary in order to build an UberTool that will take over most industries? The Japanese Fifth Generation Computing Project was a credible attempt to build such an UberTool, but it did not much rely on recursive self-improvement (apart from such things as using current computer systems to design next-generation electronics). Contrary to common misconceptions, it did not even rely on human level AI, let alone superhuman intelligence.
If this was a credible project (check the contemporary literature and you’ll find extensive discussions about its political implications and the like) why not Douglas Engelbart’s set of tools?
Why is Eliezer assuming that sustainable cycles of self-improvement are necessary in order to build an UberTool that will take over most industries? The Japanese Fifth Generation Computing Project was a credible attempt to build such an UberTool, but it did not much rely on recursive self-improvement (apart from such things as using current computer systems to design next-generation electronics). Contrary to common misconceptions, it did not even rely on human level AI, let alone superhuman intelligence.
If this was a credible project (check the contemporary literature and you’ll find extensive discussions about its political implications and the like) why not Douglas Engelbart’s set of tools?