And if interaction with such machines is the only ground you have for thinking that anything experiences consciousness, I think it would be reasonable to say that “consciousness” is whatever it is that makes those machines talk that way.
In practice, much of our notion of “consciousness” comes from observing our own mental workings, and I think we each have pretty good evidence that other people function quite similarly to ourselves, all of which makes that scenario unlikely to be the one we’re actually in.
And if interaction with such machines is the only ground you have for thinking that anything experiences consciousness, I think it would be reasonable to say that “consciousness” is whatever it is that makes those machines talk that way.
In practice, much of our notion of “consciousness” comes from observing our own mental workings, and I think we each have pretty good evidence that other people function quite similarly to ourselves, all of which makes that scenario unlikely to be the one we’re actually in.