They certainly could, but presumably they’d only be likely to if they believed that intelligence was reliably well-correlated with coming to true beliefs about the world.
If they believed instead that certain truths about the world cannot be reliably arrived at through intelligence alone, but instead require some measure of the something-else sometimes labelled “inspiration” or “Grace” or “faith” or “being properly instructed by a True Faith-holder”, they could instead believe that a superintelligent system would additionally require that something-else before it was able to recognize the truth of Mormonism. If they don’t have a good working model for what that something-else looks like, they might nevertheless decide that anything that reliably causes the system to recognize the truth of Mormonism has a better chance of making the system recognize that important truth than something that doesn’t.
Additionally, if they believed that high intelligence is not reliably adequate to overcome sufficiently wrong priors, and also believed that secular American culture embedded extremely wrong priors about (for example) the nature of God and Joseph Smith, they might decide to encode the truths about those things explicitly so as to compensate for the falsehoods already in the system.
They certainly could, but presumably they’d only be likely to if they believed that intelligence was reliably well-correlated with coming to true beliefs about the world.
If they believed instead that certain truths about the world cannot be reliably arrived at through intelligence alone, but instead require some measure of the something-else sometimes labelled “inspiration” or “Grace” or “faith” or “being properly instructed by a True Faith-holder”, they could instead believe that a superintelligent system would additionally require that something-else before it was able to recognize the truth of Mormonism. If they don’t have a good working model for what that something-else looks like, they might nevertheless decide that anything that reliably causes the system to recognize the truth of Mormonism has a better chance of making the system recognize that important truth than something that doesn’t.
Additionally, if they believed that high intelligence is not reliably adequate to overcome sufficiently wrong priors, and also believed that secular American culture embedded extremely wrong priors about (for example) the nature of God and Joseph Smith, they might decide to encode the truths about those things explicitly so as to compensate for the falsehoods already in the system.