Having LessWrong (etc.) in the corpus might actually be helpful if the chatbot is instructed to roleplay as an aligned AI (not simply an AI without any qualifiers). Then it’ll naturally imitate the behavior of an aligned AI as described in the corpus. As far as I can tell, though ChatGPT is told that it’s an AI, it’s not told that it’s an aligned AI, which seems like a missed opportunity.
(That said, for the reason of user confusion that I described in the post, I still think that it’s better to avoid the “AI” category altogether.)
That’s also a good point. I suppose I’m overextending my experience with weaker AI-ish stuff, where they tend to reproduce whatever is in their training set — regardless of whether or not it’s truly relevant.
I still think that LW would be a net disadvantage, though. If you really wanted to chuck something into an AGI and say “do this,” my current choice would be the Culture books. Maybe not optimal, but at least there’s a lot of them!
Having LessWrong (etc.) in the corpus might actually be helpful if the chatbot is instructed to roleplay as an aligned AI (not simply an AI without any qualifiers). Then it’ll naturally imitate the behavior of an aligned AI as described in the corpus. As far as I can tell, though ChatGPT is told that it’s an AI, it’s not told that it’s an aligned AI, which seems like a missed opportunity.
(That said, for the reason of user confusion that I described in the post, I still think that it’s better to avoid the “AI” category altogether.)
That’s also a good point. I suppose I’m overextending my experience with weaker AI-ish stuff, where they tend to reproduce whatever is in their training set — regardless of whether or not it’s truly relevant.
I still think that LW would be a net disadvantage, though. If you really wanted to chuck something into an AGI and say “do this,” my current choice would be the Culture books. Maybe not optimal, but at least there’s a lot of them!