Adam: Does anyone know how to X? Bella: I asked ChatGPT, and it said you Y then Z. Adam: Urgh, I could ask ChatGPT myself, why bother to speak up if you don’t have anything else to add?
I’m sort of sympathetic to Adam- I too know how to ask ChatGPT things myself- but I think he’s making a mistake. Partially because prompting good answers is a bit of a skill (one that’s becoming easier and easier as time goes on, but still a bit of a skill.) Mostly because I’m not sure if he’s reinforcing people to not answer with LLM answers, or if he’s reinforcing not telling him it’s from an LLM.
Citing sources is valuable! In grade school I sometimes got told not to use sources I found on the internet, and what happened isn’t that I stopped using the internet it’s that I stopped citing the sources that were from the internet and left them as assertions, instead adding requisite citations from other sources. It changed some of my essay structure sometimes, but not as often as I think the teachers were hoping.
Anyway, I don’t really want to encourage more people to give LLM answers when I ask questions, but if you do give me that kind of answer I appreciate it being cited as such. I try to score such that you do not lose more points telling me relevant true things vs where you don’t tell me things.
Mostly because I’m not sure if he’s reinforcing people to not answer with LLM answers, or if he’s reinforcing not telling him it’s from an LLM.
Which behavior he’s reinforcing is not up to him, it depends on the learner as well. Let’s take an analogy. Alice tells Bob “don’t steal”, and Bob interprets it as “don’t get caught stealing”. Who’s in the wrong here? Bob, of course. He’s the one choosing to ignore the intent of the request, like a misaligned AI. Same for people who misinterpret “don’t post AI slop” as “get better at passing off AI slop as human”. How such people can become genuinely aligned is a good question, and I’m not sure it can be done reliably with reinforcement, because all reinforcement has this kind of problem.
Instead of “Does anyone know how to X?” Adam could ask, “Has anyone Xed before (or recently)?” In other words, make a request for someone’s personal experience rather than for instructions.
The issue is greater when people do not have the relevant expertise to judge whether the LLM output is correct or useful. People who find things on websites generally have to evaluate the content, but several times people have responded to my questions with LLM output that plainly does not answer the question or show insight into the problem.
Possible justifications for Bella’s response in slightly different hypotheticals.
Maybe X is a good fit for an LLM. So Adam could have asked an LLM himself. Bella is politely reinforcing the behavior of checking with an LLM before asking a human.
Maybe Adam doesn’t have a subscription to a good LLM, or is away from keyboard, or doesn’t know how to use them well. Not relevant here, from Adam’s response, but Bella might not know that.
Maybe Adam is asking the question for the secondary purpose of building social bonds. Then Bella’s response achieves that objective. Compare giving Adam flowers, does he say “Urgh, I could buy flowers myself”?
I think this is part of a broader problem about asking questions and is not limited to LLM. The broader topic I’ve been thinking about a lot recently is “How to ask for help?”. The better way to ask for help often involves being specific and targeted about who you ask for help.
In this example Adam is casting a wide net, he’s not asking a domain expert on X how to do X. Casting a wide net is always going to get a lot of attempts at helpful answers form people who know nothing about X. The helpful-but-clueless to expert ratio will often increase drastically the more esoteric X is.
It’s probably pretty easy to find someone credible who knows how to cook a half-decent Spaghetti Bolognese, but, what about a Mousaka which is slightly more esoteric is going to be a bit harder. I am one of only two people in my very broad face-to-face friendship group that has ever written code in GLSL, and I’m not very good at it, so if a third friend wanted to learn about GLSL I probably won’t be a good person to ask.
I believe people like Bella are genuine in their desire and intention to help.
I also sympathize with Adam’s plight, but I think he is the problem. I sympathize because, for example, I don’t know anything about the legal structures for startup financing in my country. I wouldn’t even know if this is something that I should talk to an accountant or a lawyer about. So I understand Adam’s plight: not even knowing where to begin asking how to do X necessitates casting a wide net: going to general online communities, posting to social media, asking friends if they “know someone know who knows someone who knows how to X”. And then you’re bound to catch a lot of Bellas in that net: people genuinely trying to help, but maybe also too enthusiastic to rush in for their participation trophy by asking ChatGPT.
And the less said about people who when you ask for recommendations online give you a title of a book without any explanation about why it is relevant, why it is good, or how they expect it to help, the better. haha.
I think this dynamic more-or-less existed before ChatGPT, as well, with just plain Googling something. Especially with simpler things where the answer could be quickly found without much skill.
Even if Adam knows how to get the answer out of GPT/Google/tea leaves/etc, though, Bella is still doing something of value (it seems to me) by going through the trouble of digging around and trying to distill an answer out of the murk of digital information. Even if it only takes a couple minutes! Generally a friendly human is better at presenting the given answer, and it’s a way of signaling (at least a minimum level of) investment in helping Adam with the question. Which might even, hopefully, extend beyond a single ChatGPT query, if he’d already tried that and found its answer lacking.
I think that the way to resolve this concern is that if someone says “you Y then Z”, you ask them where they got that from, and then go look at that source. If they they say “I asked ChatGPT”, then you mock them and ignore their answer, as in your scenario. If they say “I found it on such-and-such website”, cool, you go look at that website. If they say “I have personally done this and this is my own expertise”, also cool.
In this way, the scenario where Bella got an answer from ChatGPT but doesn’t tell Adam she got it from ChatGPT can’t really happen; Adam will ask “where’d you get that”, and then either Bella tells the truth, or she lies about it but gives a real source for that answer (which is fine and is in any case not different from getting the answer from the real source to begin with), or she lies about it but gives a fake or hallucinated source for the answer (unlikely, the lie is too easily discovered), or she lies about it being her own expertise (also unlikely unless Bella is like… a sociopath or something, but this too will probably be revealed soon).
In short, make it effectively impossible to “not tell you relevant things”, and then you won’t need to worry about people possibly not telling you relevant things.
Every so often I see the following:
Adam: Does anyone know how to X?
Bella: I asked ChatGPT, and it said you Y then Z.
Adam: Urgh, I could ask ChatGPT myself, why bother to speak up if you don’t have anything else to add?
I’m sort of sympathetic to Adam- I too know how to ask ChatGPT things myself- but I think he’s making a mistake. Partially because prompting good answers is a bit of a skill (one that’s becoming easier and easier as time goes on, but still a bit of a skill.) Mostly because I’m not sure if he’s reinforcing people to not answer with LLM answers, or if he’s reinforcing not telling him it’s from an LLM.
Citing sources is valuable! In grade school I sometimes got told not to use sources I found on the internet, and what happened isn’t that I stopped using the internet it’s that I stopped citing the sources that were from the internet and left them as assertions, instead adding requisite citations from other sources. It changed some of my essay structure sometimes, but not as often as I think the teachers were hoping.
Anyway, I don’t really want to encourage more people to give LLM answers when I ask questions, but if you do give me that kind of answer I appreciate it being cited as such. I try to score such that you do not lose more points telling me relevant true things vs where you don’t tell me things.
Which behavior he’s reinforcing is not up to him, it depends on the learner as well. Let’s take an analogy. Alice tells Bob “don’t steal”, and Bob interprets it as “don’t get caught stealing”. Who’s in the wrong here? Bob, of course. He’s the one choosing to ignore the intent of the request, like a misaligned AI. Same for people who misinterpret “don’t post AI slop” as “get better at passing off AI slop as human”. How such people can become genuinely aligned is a good question, and I’m not sure it can be done reliably with reinforcement, because all reinforcement has this kind of problem.
Instead of “Does anyone know how to X?” Adam could ask, “Has anyone Xed before (or recently)?” In other words, make a request for someone’s personal experience rather than for instructions.
The issue is greater when people do not have the relevant expertise to judge whether the LLM output is correct or useful. People who find things on websites generally have to evaluate the content, but several times people have responded to my questions with LLM output that plainly does not answer the question or show insight into the problem.
Possible justifications for Bella’s response in slightly different hypotheticals.
Maybe X is a good fit for an LLM. So Adam could have asked an LLM himself. Bella is politely reinforcing the behavior of checking with an LLM before asking a human.
Maybe Adam doesn’t have a subscription to a good LLM, or is away from keyboard, or doesn’t know how to use them well. Not relevant here, from Adam’s response, but Bella might not know that.
Maybe Adam is asking the question for the secondary purpose of building social bonds. Then Bella’s response achieves that objective. Compare giving Adam flowers, does he say “Urgh, I could buy flowers myself”?
I think this is part of a broader problem about asking questions and is not limited to LLM. The broader topic I’ve been thinking about a lot recently is “How to ask for help?”. The better way to ask for help often involves being specific and targeted about who you ask for help.
In this example Adam is casting a wide net, he’s not asking a domain expert on X how to do X. Casting a wide net is always going to get a lot of attempts at helpful answers form people who know nothing about X. The helpful-but-clueless to expert ratio will often increase drastically the more esoteric X is.
It’s probably pretty easy to find someone credible who knows how to cook a half-decent Spaghetti Bolognese, but, what about a Mousaka which is slightly more esoteric is going to be a bit harder. I am one of only two people in my very broad face-to-face friendship group that has ever written code in GLSL, and I’m not very good at it, so if a third friend wanted to learn about GLSL I probably won’t be a good person to ask.
I believe people like Bella are genuine in their desire and intention to help.
I also sympathize with Adam’s plight, but I think he is the problem. I sympathize because, for example, I don’t know anything about the legal structures for startup financing in my country. I wouldn’t even know if this is something that I should talk to an accountant or a lawyer about. So I understand Adam’s plight: not even knowing where to begin asking how to do X necessitates casting a wide net: going to general online communities, posting to social media, asking friends if they “know someone know who knows someone who knows how to X”. And then you’re bound to catch a lot of Bellas in that net: people genuinely trying to help, but maybe also too enthusiastic to rush in for their participation trophy by asking ChatGPT.
And the less said about people who when you ask for recommendations online give you a title of a book without any explanation about why it is relevant, why it is good, or how they expect it to help, the better. haha.
I think this dynamic more-or-less existed before ChatGPT, as well, with just plain Googling something. Especially with simpler things where the answer could be quickly found without much skill.
Even if Adam knows how to get the answer out of GPT/Google/tea leaves/etc, though, Bella is still doing something of value (it seems to me) by going through the trouble of digging around and trying to distill an answer out of the murk of digital information. Even if it only takes a couple minutes! Generally a friendly human is better at presenting the given answer, and it’s a way of signaling (at least a minimum level of) investment in helping Adam with the question. Which might even, hopefully, extend beyond a single ChatGPT query, if he’d already tried that and found its answer lacking.
So, I agree that Adam is making a mistake.
I think that the way to resolve this concern is that if someone says “you Y then Z”, you ask them where they got that from, and then go look at that source. If they they say “I asked ChatGPT”, then you mock them and ignore their answer, as in your scenario. If they say “I found it on such-and-such website”, cool, you go look at that website. If they say “I have personally done this and this is my own expertise”, also cool.
In this way, the scenario where Bella got an answer from ChatGPT but doesn’t tell Adam she got it from ChatGPT can’t really happen; Adam will ask “where’d you get that”, and then either Bella tells the truth, or she lies about it but gives a real source for that answer (which is fine and is in any case not different from getting the answer from the real source to begin with), or she lies about it but gives a fake or hallucinated source for the answer (unlikely, the lie is too easily discovered), or she lies about it being her own expertise (also unlikely unless Bella is like… a sociopath or something, but this too will probably be revealed soon).
In short, make it effectively impossible to “not tell you relevant things”, and then you won’t need to worry about people possibly not telling you relevant things.