Because then we can’t trust that that’s what the moons of Neptune really look like. The information has come from a source with goals and motivations and long-term plans, and the ability to lie. If a space probe tells us that the largest moon of Neptune has black geysers and terrain shaped like cantaloupe skin, we can trust it because it’s subhuman and incapable of fooling us. With an AI we have to think “what if it’s wrong? What if it has an ulterior motive?”
It occurs to me that both of my examples are similar, in that the moon of Neptune are remote in space, while historical facts are remote in time. We can imagine facts that are both. A few years ago, the comet Omuamua passed through the solar system on an interstellar journey. We took lots of observations of its weird properties as it passed the Sun, and then it vanished back into the interstellar darkness. The longer we wait, the harder it would be to send a space probe.
Because then we can’t trust that that’s what the moons of Neptune really look like. The information has come from a source with goals and motivations and long-term plans, and the ability to lie. If a space probe tells us that the largest moon of Neptune has black geysers and terrain shaped like cantaloupe skin, we can trust it because it’s subhuman and incapable of fooling us. With an AI we have to think “what if it’s wrong? What if it has an ulterior motive?”
It occurs to me that both of my examples are similar, in that the moon of Neptune are remote in space, while historical facts are remote in time. We can imagine facts that are both. A few years ago, the comet Omuamua passed through the solar system on an interstellar journey. We took lots of observations of its weird properties as it passed the Sun, and then it vanished back into the interstellar darkness. The longer we wait, the harder it would be to send a space probe.