To be clear—GPT-4 is the entity that said “that’s a fair point.” :P
I’m aware that ‘chatbots’ have existed for half a century, and that LLMs are more than just a chatbot. But are you asserting that there are conscious chatbots right now? Sure, LLMs represent a step-change in the ability of ‘chatbots’. But I’m not sure that it’s a category error if there in fact are no conscious chatbots at this time.
Yair, you are correct.
Point 2) is why I wrote the story. In a conversation about the potential for AI rights, some friends and I came to the disconcerting conclusion that it’s kinda impossible to justify your own consciousness (to other people). That unnerving thought prompted the story, since if we ourselves can’t justify our consciousness, how can we reasonably expect an AI to do so?