Why does this make you more bearish on vibes? The reason I ask is that I think of “vibes” as aggregating over a much wider (but siloed) social network and a lot more sources of information. It would be interesting to know about to what extent rigorous high-n survey methods would reveal discrepancies between assumptions and reality about people’s perceptions in this and other areas to do with pressing social issues.
like, suppose i anecdotally noticed a few people last year be visibly confused when i said the phrase AGI in normal conversation last year, and then this year i noticed that many fewer people were visibly confused by AGI. then, this would tell me almost nothing about whether name-recognition of AGI increased or decreased; at n=10, it is nearly impossible to say anything whatsoever.
What’s your reasoning or assumptions for why it would tell you ~nothing to witness name recognition increasing like that? I’m assuming it’s not just because “visible confusion” isn’t a perfect proxy for lack of name recognition (and vice versa).
I guess I’m more bullish on vibes being a more powerful way to gauge name recognition than you seem to be. So here is a toy model to explain why. ChatGPT was released in Nov. 2022, so prior to that we can say it had approximately zero name recognition.
I’ve never read a survey on ChatGPT name recognition. I have only my anecdotal evidence to go on. But I am extremely confident based on what I think can fairly be called “vibes” that ChatGPT has massive name recognition, at least in America, compared to November 2022. If there was a reliable way to test this, I’d be willing to wager big money on it, provided I could feel confident in my ability to truly pin a number on what I mean by “massive name recognition.” Do you agree with this model but feel bearish on vibes more generally? Are you skeptical of my perception that ChatGPT’s name recognition has exploded since its release?
sure, you can notice extremely large effect sizes through vibes. but the claim is that for even “smaller” effect sizes (like, tens of percentage points, e.g 50->75%), you need pretty big sample sizes. obviously 0->100% doesn’t need a very large sample size.
I agree that chatgpt obviously has lots of name recognition but I do also separately think chatgpt has less name recognition than you might guess. I predict that only 85% of Americans would get a multiple choice question right about what kind of app chatgpt is (choices: artificial intelligence; social media; messaging and calling; online dating). whereas a control question about e.g Google will get like 97% or whatever the lizardman constant dictates
Reasonable, I also don’t expect that I could pick up on a 1.5x increase in name recognition over a year based on vibes—didn’t read closely enough to notice you were talking about a 10% increase, so sorry about the time waste.
Why does this make you more bearish on vibes? The reason I ask is that I think of “vibes” as aggregating over a much wider (but siloed) social network and a lot more sources of information. It would be interesting to know about to what extent rigorous high-n survey methods would reveal discrepancies between assumptions and reality about people’s perceptions in this and other areas to do with pressing social issues.
like, suppose i anecdotally noticed a few people last year be visibly confused when i said the phrase AGI in normal conversation last year, and then this year i noticed that many fewer people were visibly confused by AGI. then, this would tell me almost nothing about whether name-recognition of AGI increased or decreased; at n=10, it is nearly impossible to say anything whatsoever.
What’s your reasoning or assumptions for why it would tell you ~nothing to witness name recognition increasing like that? I’m assuming it’s not just because “visible confusion” isn’t a perfect proxy for lack of name recognition (and vice versa).
I guess I’m more bullish on vibes being a more powerful way to gauge name recognition than you seem to be. So here is a toy model to explain why. ChatGPT was released in Nov. 2022, so prior to that we can say it had approximately zero name recognition.
I’ve never read a survey on ChatGPT name recognition. I have only my anecdotal evidence to go on. But I am extremely confident based on what I think can fairly be called “vibes” that ChatGPT has massive name recognition, at least in America, compared to November 2022. If there was a reliable way to test this, I’d be willing to wager big money on it, provided I could feel confident in my ability to truly pin a number on what I mean by “massive name recognition.” Do you agree with this model but feel bearish on vibes more generally? Are you skeptical of my perception that ChatGPT’s name recognition has exploded since its release?
sure, you can notice extremely large effect sizes through vibes. but the claim is that for even “smaller” effect sizes (like, tens of percentage points, e.g 50->75%), you need pretty big sample sizes. obviously 0->100% doesn’t need a very large sample size.
I agree that chatgpt obviously has lots of name recognition but I do also separately think chatgpt has less name recognition than you might guess. I predict that only 85% of Americans would get a multiple choice question right about what kind of app chatgpt is (choices: artificial intelligence; social media; messaging and calling; online dating). whereas a control question about e.g Google will get like 97% or whatever the lizardman constant dictates
Reasonable, I also don’t expect that I could pick up on a 1.5x increase in name recognition over a year based on vibes—didn’t read closely enough to notice you were talking about a 10% increase, so sorry about the time waste.