That’s an appealing thought, but I’m not sure it’s a true one.
For one, if we’re talking about appealing to general audiences, many folk won’t be trying to develop an AGI, but still be relevant to our interests. Thinking AGI can not invent because they lack souls, or that AGI will be friendly if annoying golden translation droids, may be inconsistent with writing evolutionary algorithms, but is not certainly inconsistent with having investment or political capital.
At a deeper level, a lot of folk do hold such beliefs and simultaneously have inconsistent belief structures, which may still leave them dangerous. It is demonstrably possible have incorrect beliefs about evolution yet run a PCR, or to think it’s easy to preserve semantic significance but also be a computer programmer. It’s tempting to dismiss people who hold irrational beliefs since rationality strongly correlates with long-term success, but from an absolute safety perspective that gets increasingly risky.
You need a bit more to develop an AGI than running PCR that someone else invents. I don’t think you can develop an AGI when you think AGI are impossible due to metaphysical dualism.
You can believe that humans have souls are still design AGI that have minds but no souls, but you won’t get far at developing an AGI with something like a mind if you think that task is impossible.
That’s an appealing thought, but I’m not sure it’s a true one.
For one, if we’re talking about appealing to general audiences, many folk won’t be trying to develop an AGI, but still be relevant to our interests. Thinking AGI can not invent because they lack souls, or that AGI will be friendly if annoying golden translation droids, may be inconsistent with writing evolutionary algorithms, but is not certainly inconsistent with having investment or political capital.
At a deeper level, a lot of folk do hold such beliefs and simultaneously have inconsistent belief structures, which may still leave them dangerous. It is demonstrably possible have incorrect beliefs about evolution yet run a PCR, or to think it’s easy to preserve semantic significance but also be a computer programmer. It’s tempting to dismiss people who hold irrational beliefs since rationality strongly correlates with long-term success, but from an absolute safety perspective that gets increasingly risky.
You need a bit more to develop an AGI than running PCR that someone else invents. I don’t think you can develop an AGI when you think AGI are impossible due to metaphysical dualism.
You can believe that humans have souls are still design AGI that have minds but no souls, but you won’t get far at developing an AGI with something like a mind if you think that task is impossible.