In a similar vein, a solid moral-philosophical grounding on what the stakes of power are and why they matter are a must-have. Too many AI safety researchers are desperately naive about the game of power and about the overall sweep of sociotechnical development.
It seems to me that this was already explored in enough detail and that our goal, conditioned on solving technical alignment, is to convince decisionmakers to implement the solution. The TLDR of the post-alignment problem is that mankind will be rendered unable to meaningfully impact the economy because decisionmakers will realize that any cognitive task is far cheaper to outsource to the AIs, ensuring that mankind can no longer obtain resources by any form of labor. The closest thing to a solution of the problem would be implementing some combination of MacAskell’s Universal Basic Resources combined with games with clear rules and ability to earn prizes.
As for solving technical alignment itself, I understand how math-related books or the security mindset could help, but I struggle to understand how exactly governance-related ideas or sections related to game theory and coordination problems can help, with the exception of making deals with not-so-aligned AIs.
There is also an additional consideration that rogue or careless (think of xAI’s lack of precautions) AGI development should be prevented until alignment is solved or forever if alignment is deemed insoluble, but I suspect that it also is easily understandable. Therefore, I cannot see a causal pathway for reading about governance to become useful for AI safety.
It seems to me that this was already explored in enough detail and that our goal, conditioned on solving technical alignment, is to convince decisionmakers to implement the solution. The TLDR of the post-alignment problem is that mankind will be rendered unable to meaningfully impact the economy because decisionmakers will realize that any cognitive task is far cheaper to outsource to the AIs, ensuring that mankind can no longer obtain resources by any form of labor. The closest thing to a solution of the problem would be implementing some combination of MacAskell’s Universal Basic Resources combined with games with clear rules and ability to earn prizes.
As for solving technical alignment itself, I understand how math-related books or the security mindset could help, but I struggle to understand how exactly governance-related ideas or sections related to game theory and coordination problems can help, with the exception of making deals with not-so-aligned AIs.
There is also an additional consideration that rogue or careless (think of xAI’s lack of precautions) AGI development should be prevented until alignment is solved or forever if alignment is deemed insoluble, but I suspect that it also is easily understandable. Therefore, I cannot see a causal pathway for reading about governance to become useful for AI safety.