When you talk about ‘black-box’ versions of Hugh, do you envision that H is able to answer questions relating to the cognitive processes that lead to the answer given, or about H’s thinking in general? This seems to contradict the spirit of a black box but self reflection is an important part of Hugh’s cognitive ability.
Perhaps they are both useful possibilities, my intuition is that this kind of self reflection is as far from being possible for AI as any human ability and so we should expect that we might have systems powerful enough to take on wide responsibility without this ability. If it were possible, though, the ability to use loops of self reflection to check whether a cognitive process serves a certain goal would be very helpful.
When you talk about ‘black-box’ versions of Hugh, do you envision that H is able to answer questions relating to the cognitive processes that lead to the answer given, or about H’s thinking in general? This seems to contradict the spirit of a black box but self reflection is an important part of Hugh’s cognitive ability.
Perhaps they are both useful possibilities, my intuition is that this kind of self reflection is as far from being possible for AI as any human ability and so we should expect that we might have systems powerful enough to take on wide responsibility without this ability. If it were possible, though, the ability to use loops of self reflection to check whether a cognitive process serves a certain goal would be very helpful.