Great overview! And thanks for linking to Bostrom’s paper on OAI, I hadn’t read that yet.
My immediate thoughts: Isn’t it most likely that advanced AI will be used by humans to advance any of their personal goals (benign or not benign). Therefore, the phrase “alignment to human values” automatically raises the question, who’s values? Which leads to: who gets to decide what values it gets aligned to?
Great overview! And thanks for linking to Bostrom’s paper on OAI, I hadn’t read that yet.
My immediate thoughts: Isn’t it most likely that advanced AI will be used by humans to advance any of their personal goals (benign or not benign). Therefore, the phrase “alignment to human values” automatically raises the question, who’s values? Which leads to: who gets to decide what values it gets aligned to?