On AI: are we sure we are not influenced by meta-religious ideas of sci-fi writers who write about sufficiently advanced computers just “waking up into consciousness” i.e. create a hard, almost soul-like, barrier between conscious and not conscious, which carries an assumption that consciousness is a typically human-like feature? It is meta-religious as it is based on the unique specialness of the human soul.
I mean I think the potential variation space of intelligent, conscious agents is very, very large and a randomly selected AI will not be human-like in way we would recognize it to. We will not recognize its consciousness, we will not recognize its intelligence, even its agency, all we would see it does mysterious complicated stuff we don’t understand. It may almost look random. It does stuff, maybe it communicates with us although the human-language words it uses will not reflects its thought processes, but it will be profoundly alien.
I think the thought-process of AI is expected to be alien by anyone who take AGI seriously. It’s just not all that relevant to discussions about the threats and possibilities about it.
On AI: are we sure we are not influenced by meta-religious ideas of sci-fi writers who write about sufficiently advanced computers just “waking up into consciousness” i.e. create a hard, almost soul-like, barrier between conscious and not conscious, which carries an assumption that consciousness is a typically human-like feature? It is meta-religious as it is based on the unique specialness of the human soul.
I mean I think the potential variation space of intelligent, conscious agents is very, very large and a randomly selected AI will not be human-like in way we would recognize it to. We will not recognize its consciousness, we will not recognize its intelligence, even its agency, all we would see it does mysterious complicated stuff we don’t understand. It may almost look random. It does stuff, maybe it communicates with us although the human-language words it uses will not reflects its thought processes, but it will be profoundly alien.
I think the thought-process of AI is expected to be alien by anyone who take AGI seriously. It’s just not all that relevant to discussions about the threats and possibilities about it.
Related article.