Thanks! The linked article is exactly what I was looking for. Assuming “nationalization” means something like “soft nationalization” does make the timeline seem a lot more plausible.
mushroomsoup
Karma: 13
[Translation] In the Age of AI don’t Look for Unicorns
[Translation] AI Generated Fake News is Taking Over my Family Group Chat
How are Those AI Participants Doing Anyway?
According to the timeline of the post, AGI will take place during the Trump presidency and much of the nationalization efforts will need to be lead by his administration. However, that seems antithetical to the administrations general ethos of deregulation (at least in the banking and energy sectors). Would it be possible to explain which avenues would lead Trump to nationalize, for example, OpenAI or Antropic?
This comment raises some good points, but even “there will be a natural pressure for [subprocesses] to resemble a corrigible agent” seem to be debatable. Again consider the restaurant setting. It is sometime necessary for restaurants to close temporarily for renovation to increase the seating capacity, upgrade equipment, etc. The head chef who decided to renovate will be making the instrumental goals of all the other chefs (make a good food, earn money to stay alive) untenable while they are furloughed. More generally, progress towards terminal goals is not monotonic and thus only focusing on the local topology of the optimization landscape might be insufficient to predict long-horizon trends.