I think there’s another danger worth considering here, which is that AI power can be expected to increase as we approach AGI. Depending on how long we are stuck in the pre-AGI regime, and how powerful AI had become in that time, there is a danger that unethical humans substantially empowered by AI tools may attempt to gain and maintain power over society, and that part of this action may be interfering with labs attempting to develop aligned AGI. I feel like this possibility casts the potential race dynamics in a different light.
Here’s a post discussion the potential wide reaching effects of social manipulation via powerful but pre-AGI tech: https://www.lesswrong.com/posts/3broJA5XpBwDbjsYb/agency-engineering-is-ai-alignment-to-human-intent-enough
I think there’s another danger worth considering here, which is that AI power can be expected to increase as we approach AGI. Depending on how long we are stuck in the pre-AGI regime, and how powerful AI had become in that time, there is a danger that unethical humans substantially empowered by AI tools may attempt to gain and maintain power over society, and that part of this action may be interfering with labs attempting to develop aligned AGI. I feel like this possibility casts the potential race dynamics in a different light. Here’s a post discussion the potential wide reaching effects of social manipulation via powerful but pre-AGI tech: https://www.lesswrong.com/posts/3broJA5XpBwDbjsYb/agency-engineering-is-ai-alignment-to-human-intent-enough