By the way, you’re making an awful lot of extremely strong and very common points with no evidence here (“ChaosGPT is aligned”, “we know how to ensure alignment”, “the AI understanding that you don’t want it to destroy humanity implies that it will not want to destroy humanity”, “the AI will refuse to cooperate with people who have ill intentions”, “a system that optimises a loss function and approximates a data generation function will highly value human life by default”, “a slight misalignment is far from doomsday”, “an entity that is built to maximise something might doubt its mission”), as well as the standard “it’s better to focus on X than Y” in an area where almost nobody is focusing on Y anyway. What’s your background, so that we can recommend the appropriate reading material? For example, have you read the Sequences, or Bostrom’s Superintelligence?
By the way, you’re making an awful lot of extremely strong and very common points with no evidence here (“ChaosGPT is aligned”, “we know how to ensure alignment”, “the AI understanding that you don’t want it to destroy humanity implies that it will not want to destroy humanity”, “the AI will refuse to cooperate with people who have ill intentions”, “a system that optimises a loss function and approximates a data generation function will highly value human life by default”, “a slight misalignment is far from doomsday”, “an entity that is built to maximise something might doubt its mission”), as well as the standard “it’s better to focus on X than Y” in an area where almost nobody is focusing on Y anyway. What’s your background, so that we can recommend the appropriate reading material? For example, have you read the Sequences, or Bostrom’s Superintelligence?