What about narrow-domain AI? If, through neuroscience, we figure out more about how the brain comes up with and generalizes theories about, say, physics and math, couldn’t we emulate just this part, so that we get a complicated calculator/theory generator that can’t modify itself or do much else? Then it would only use the computing power we provide it with (won’t turn the Earth into a computer) and we could use it to help prove Friendliness-related theorems or explore math/physics in general (on much faster timescales than today).
Theorem-proving is a narrow domain that operates entirely within an abstract logic, and so is probably completely safe. Theorem-generating would benefit from more general cognitive abilities, so is more open-ended and potentially dangerous.
What about narrow-domain AI? If, through neuroscience, we figure out more about how the brain comes up with and generalizes theories about, say, physics and math, couldn’t we emulate just this part, so that we get a complicated calculator/theory generator that can’t modify itself or do much else? Then it would only use the computing power we provide it with (won’t turn the Earth into a computer) and we could use it to help prove Friendliness-related theorems or explore math/physics in general (on much faster timescales than today).
Theorem-proving is a narrow domain that operates entirely within an abstract logic, and so is probably completely safe. Theorem-generating would benefit from more general cognitive abilities, so is more open-ended and potentially dangerous.