This. “Orthogonality” is just the name the rationality/AI safety community gave to the old philosophical adage that “you can not derive ought from is”. No matter how smart you are at understanding the world, none of that will make you intrinsically good. Morality isn’t inscribed somewhere in the mathematical structure of spacetime like the speed of light or Planck’s constant. The universe just doesn’t care; and thus no amounting of understanding the universe will in itself make you care.
This. “Orthogonality” is just the name the rationality/AI safety community gave to the old philosophical adage that “you can not derive ought from is”. No matter how smart you are at understanding the world, none of that will make you intrinsically good. Morality isn’t inscribed somewhere in the mathematical structure of spacetime like the speed of light or Planck’s constant. The universe just doesn’t care; and thus no amounting of understanding the universe will in itself make you care.