Well the idea, I gather, is that the FAI will enforce that.
In practice though, it has not even been explicitly listed as requirement for powerful AGI (let alone articulated why), that the AGI has to, on same hardware, outperform—at the task of designing better computers for example—the non-general-intelligent tools which work by combination of iteration with analytic methods and hill climbing (and perhaps better approaches taken from the AGI effort), and whose problem and solution spaces are well defined. (those can think outside the box in human terms, as them search bigger spaces than humans, but within the permissible model and solution space, i.e. ‘killing humans helps make better microchip’ sort of statements are not even processable in the system which is designed for engineering materials at microscale)
Well the idea, I gather, is that the FAI will enforce that.
In practice though, it has not even been explicitly listed as requirement for powerful AGI (let alone articulated why), that the AGI has to, on same hardware, outperform—at the task of designing better computers for example—the non-general-intelligent tools which work by combination of iteration with analytic methods and hill climbing (and perhaps better approaches taken from the AGI effort), and whose problem and solution spaces are well defined. (those can think outside the box in human terms, as them search bigger spaces than humans, but within the permissible model and solution space, i.e. ‘killing humans helps make better microchip’ sort of statements are not even processable in the system which is designed for engineering materials at microscale)