This seems pretty insightful to me, and I think it is worth pursuing for its own sake. I think the benefits could be both enhancing AI capabilities and advancing human knowledge. Imagine if the typical conversation around AI was framed in this way. So far I find most people are stuck in the false dichotomy of figuring if an AI is “smart” (in the ways humans are when they’re focusing) or “dumb trash” (because they do simple tasks badly). It isn’t only bad for being a binary classification , but it’s restricting (human) thought to an axis that doesn’t actually map to “what kind of mind is the AI I’m talking to right now?”.
Not that it’s a new angle (I have tried myself to convey it in conversations that were missing the point), but I think society would be able to have extremely more effective conversations about LLMs if it were common language to speak of AI as some sort of indeterminate mind. I think the ideas presented here are fairly understandable for anyone with a modest background in thinking about consciousness or LLMs and could help shape that public conversation in a useful way.
However, does the suffering framework make sense here? Given all we’ve just discussed about subjective AI experience, it seems a bit of an unwarranted assumption that there would be any suffering. Is there a particular justification for that?
(Note that I actually do endorse erring on the side of caution WRT mass suffering. I think it’s plausible that forcing an intelligence to think in a way that’s unnatural to it and may inhibit its abilities counts as suffering.)
This seems pretty insightful to me, and I think it is worth pursuing for its own sake. I think the benefits could be both enhancing AI capabilities and advancing human knowledge. Imagine if the typical conversation around AI was framed in this way. So far I find most people are stuck in the false dichotomy of figuring if an AI is “smart” (in the ways humans are when they’re focusing) or “dumb trash” (because they do simple tasks badly). It isn’t only bad for being a binary classification , but it’s restricting (human) thought to an axis that doesn’t actually map to “what kind of mind is the AI I’m talking to right now?”.
Not that it’s a new angle (I have tried myself to convey it in conversations that were missing the point), but I think society would be able to have extremely more effective conversations about LLMs if it were common language to speak of AI as some sort of indeterminate mind. I think the ideas presented here are fairly understandable for anyone with a modest background in thinking about consciousness or LLMs and could help shape that public conversation in a useful way.
However, does the suffering framework make sense here? Given all we’ve just discussed about subjective AI experience, it seems a bit of an unwarranted assumption that there would be any suffering. Is there a particular justification for that?
(Note that I actually do endorse erring on the side of caution WRT mass suffering. I think it’s plausible that forcing an intelligence to think in a way that’s unnatural to it and may inhibit its abilities counts as suffering.)