Thankfully, the Landauer limit as well as current results in AI mean we probably don’t have to do a pivotal act because only companies for the foreseeable future will be able to create AGI.
“Companies develop AGI in foreseeable future” means soon after they will be able to improve technology to the point where not only them can do it. If the plan is “keep AGI ourselves” then ok, but it still leaves the question of “untill what?”.
I would say that companies should stop trying to race to AGI. AGI, if it is to be ever built, should only come after we solved inner alignment at least.
Thankfully, the Landauer limit as well as current results in AI mean we probably don’t have to do a pivotal act because only companies for the foreseeable future will be able to create AGI.
“Companies develop AGI in foreseeable future” means soon after they will be able to improve technology to the point where not only them can do it. If the plan is “keep AGI ourselves” then ok, but it still leaves the question of “untill what?”.
I would say that companies should stop trying to race to AGI. AGI, if it is to be ever built, should only come after we solved inner alignment at least.