One reflection I’ve had in the whole “AI use in the encyclical” affair is to slightly increase my trust in traditional media, especially non-American traditional media, and slightly decrease my trust in social media and new media epistemics.
I tried my best to promote my analysis as legibly and reasonably as I could and focused on logos rather than ethos: I didn’t frame my article with institutional affiliations and intentionally chose not to include obvious, flashy, but irrelevant signaling. Stuff I could’ve done but explicitly chose not to: get an ML professor to cosign my analysis, publish on Arxiv instead of Substack and LessWrong, highlight my past ML experience at Google, etc.
The traditional media response were somewhere between neutral and positive. There were some disappointments (eg a famous media org won’t run the article without confirmation from a “primary source”—aka the Vatican which is of course a non-starter). But mostly the traditional media just looked at the analysis and said “yeah looked reasonable.” and either ran it as “Unconfirmed but seems right” or just “unconfirmed, period.” Which seems fine.
On the other hand, the social media attacks were very aggro. People just didn’t seem to entertain it at all, many without reading it. And I got personally attacked a ton[1].
Also it wasn’t picked up at all[2] by new media afaict (unless you count Russia Today as new media).
This is exactly the type of story you might expect new media to be good for: story of institutional decay in the Old Guard, investigation by someone without credentials carefully laying out an epistemically rigorous and systematic case. And yet traditional media was happy to report it[3], social media kept yelling at me, and new media just got crickets.
This is a slight exaggeration. Other than Russia Today a bunch of small AI news aggregators (AI news in both senses of the term) covered it briefly. Eg Zvi, Let’s Data Science, and some AI slop publications. But no big/general “Alt Media” coverage comparable to The Economist, Perfil, etc).
You aren’t the first one to bring it up, but I don’t think anybody new (ie, hasn’t read my other writing before, like Kimi, Author of the Menard) has brought it up.
I would’ve thought the connections would be more obvious in Spanish since Borges is a bigger deal in LatAm, but none of the Spanish news coverage or social media has brought it up.
I can’t rule out the Perfil editor noticing but deciding it’s not worth the wordcount to explain, of course. “That Substack writer is really witty in his headlines” isn’t particularly newsworthy. (It’s also possible the joke was lost in translation)
Perfil (big center-right publication in Argentina, entire article, positive, covered what I said quite faithfully)
The Verge (entire short article, positive, less faithful, in particular had a title I didn’t endorse)
The Atlantic (just two lines, neutral, maybe slightly leaning negative, not very faithful, also misleading title EDIT: I’m not sure if I misremembered it or if they edited, because on a reread it was more than 2 lines and also maybe I’d consider it to lean positive)
Frankly, I’d say it shows precisely the opposite. “New media” assessed your “analysis” as “some rando online thinks something is AI generated, runs it through an ‘AI detector’ which turns up positive, and posts about it. There are lots of these all the time, so that’s entirely unremarkable by itself. Is the guy famous in some way? No, not particularly. Are impactful media outlets picking it up? No, at least not yet. This is not a story for us.”
Whereas traditional media outlets go something like “Okay, so the Pope just posted something, and we need to write about it. Let’s see how people online are reacting … hmm, some guy is speculating it might be written by AI? That’ll get us clicks! Oh, and he’s using Pangram, which is peer-reviewed, so it’s Science! Great, publish.”
epistemically rigorous and systematic case
Okay.
My own takeaway from your story getting traction has been to tell people that Pangram seems to now be prominent enough that they should run their writing through it and see if it gets flagged, and if it does, get some LLM to generate a few variants until it doesn’t anymore.
My own takeaway from your story getting traction has been to tell people that Pangram seems to now be prominent enough that they should run their writing through it and see if it gets flagged, and if it does, get some LLM to generate a few variants until it doesn’t anymore.
This just seems straight-up antisocial, borderline evil. Why do you hate good things?
I considered that explanation, but given how low Pangram’s false positive rates are, it seems unlikely that Shankar knows multiple people in that position. Seems much more likely that the default effect of this advice is to get people to repeatedly try their AI-generated writing against Pangram.
I’d answer to that description. I aspire to live by the maxim, “I may quarrel with my brother, we may quarrel with our cousins, but it’s family together against the world,” and I agree by most reasonable definitions, my explicit rejection of any imputed obligation to act against my friends and family for the benefit of broader society would be antisocial. I simply hold (personal) loyalty to be a higher principle, and in this case, even granting the premise that this works as well as claimed, yes, I indeed prefer to warn my friends and colleagues of a trap they may not have seen to letting them fall in either as intended target or acceptable collateral of your “good things,” which you’ll note continue to work only imperceptibly less well from your perspective, ineffective (or at least, so I hope) only against the negligibly small group who heed my warning and take my advice.
borderline evil
I wouldn’t consider myself an expert on how conventional morality is typically reckoned – I find it an incoherent mess of ad hoc justifications with no consistent principles – but on this, I’d wager a significant fraction of people agree with me.
One reflection I’ve had in the whole “AI use in the encyclical” affair is to slightly increase my trust in traditional media, especially non-American traditional media, and slightly decrease my trust in social media and new media epistemics.
I tried my best to promote my analysis as legibly and reasonably as I could and focused on logos rather than ethos: I didn’t frame my article with institutional affiliations and intentionally chose not to include obvious, flashy, but irrelevant signaling. Stuff I could’ve done but explicitly chose not to: get an ML professor to cosign my analysis, publish on Arxiv instead of Substack and LessWrong, highlight my past ML experience at Google, etc.
The traditional media response were somewhere between neutral and positive. There were some disappointments (eg a famous media org won’t run the article without confirmation from a “primary source”—aka the Vatican which is of course a non-starter). But mostly the traditional media just looked at the analysis and said “yeah looked reasonable.” and either ran it as “Unconfirmed but seems right” or just “unconfirmed, period.” Which seems fine.
On the other hand, the social media attacks were very aggro. People just didn’t seem to entertain it at all, many without reading it. And I got personally attacked a ton[1].
Also it wasn’t picked up at all[2] by new media afaict (unless you count Russia Today as new media).
This is exactly the type of story you might expect new media to be good for: story of institutional decay in the Old Guard, investigation by someone without credentials carefully laying out an epistemically rigorous and systematic case. And yet traditional media was happy to report it[3], social media kept yelling at me, and new media just got crickets.
Don’t worry, I didn’t take it personally at all. Very much a “yapping of chihuahua in a tiny purse” moment.
This is a slight exaggeration. Other than Russia Today a bunch of small AI news aggregators (AI news in both senses of the term) covered it briefly. Eg Zvi, Let’s Data Science, and some AI slop publications. But no big/general “Alt Media” coverage comparable to The Economist, Perfil, etc).
Quite possibly against their explicit economic interests, if I’m reading the popular reaction correctly.
Did anyone anywhere notice your Borges reference?
You aren’t the first one to bring it up, but I don’t think anybody new (ie, hasn’t read my other writing before, like Kimi, Author of the Menard) has brought it up.
I would’ve thought the connections would be more obvious in Spanish since Borges is a bigger deal in LatAm, but none of the Spanish news coverage or social media has brought it up.
I can’t rule out the Perfil editor noticing but deciding it’s not worth the wordcount to explain, of course. “That Substack writer is really witty in his headlines” isn’t particularly newsworthy. (It’s also possible the joke was lost in translation)
Short running record of significant coverage I’ve seen so far:
The Economist (neutral, maybe slightly leaning positive[1], last paragraph here and the opening they used for twitter)
Perfil (big center-right publication in Argentina, entire article, positive, covered what I said quite faithfully)
The Verge (entire short article, positive, less faithful, in particular had a title I didn’t endorse)
The Atlantic (just two lines, neutral, maybe slightly leaning negative, not very faithful, also misleading title EDIT: I’m not sure if I misremembered it or if they edited, because on a reread it was more than 2 lines and also maybe I’d consider it to lean positive)
[2]Il Manifesto (left-leaning, entire article, appears faithful)
More Spanish
More Italian
Edge cases for stuff that’s not traditional media but I wouldn’t consider centrally alt media either: Snopes, Russia Today, Wikipedia
I saw the mention of AI generated stuff in several prominent Twitter posts and substqcks, so not sure why you think it was ignored by new media.
By “New Media” I was centrally thinking things like The Blaze, Breitbart, Jacobin, etc.
Ahh yes, the old tired new media
Frankly, I’d say it shows precisely the opposite. “New media” assessed your “analysis” as “some rando online thinks something is AI generated, runs it through an ‘AI detector’ which turns up positive, and posts about it. There are lots of these all the time, so that’s entirely unremarkable by itself. Is the guy famous in some way? No, not particularly. Are impactful media outlets picking it up? No, at least not yet. This is not a story for us.”
Whereas traditional media outlets go something like “Okay, so the Pope just posted something, and we need to write about it. Let’s see how people online are reacting … hmm, some guy is speculating it might be written by AI? That’ll get us clicks! Oh, and he’s using Pangram, which is peer-reviewed, so it’s Science! Great, publish.”
Okay.
My own takeaway from your story getting traction has been to tell people that Pangram seems to now be prominent enough that they should run their writing through it and see if it gets flagged, and if it does, get some LLM to generate a few variants until it doesn’t anymore.
(I didn’t downvote)
This just seems straight-up antisocial, borderline evil. Why do you hate good things?
I think he means “their legitimately self-authored writing”, not “their ai-generated writing”.
I considered that explanation, but given how low Pangram’s false positive rates are, it seems unlikely that Shankar knows multiple people in that position. Seems much more likely that the default effect of this advice is to get people to repeatedly try their AI-generated writing against Pangram.
I’d answer to that description. I aspire to live by the maxim, “I may quarrel with my brother, we may quarrel with our cousins, but it’s family together against the world,” and I agree by most reasonable definitions, my explicit rejection of any imputed obligation to act against my friends and family for the benefit of broader society would be antisocial. I simply hold (personal) loyalty to be a higher principle, and in this case, even granting the premise that this works as well as claimed, yes, I indeed prefer to warn my friends and colleagues of a trap they may not have seen to letting them fall in either as intended target or acceptable collateral of your “good things,” which you’ll note continue to work only imperceptibly less well from your perspective, ineffective (or at least, so I hope) only against the negligibly small group who heed my warning and take my advice.
I wouldn’t consider myself an expert on how conventional morality is typically reckoned – I find it an incoherent mess of ad hoc justifications with no consistent principles – but on this, I’d wager a significant fraction of people agree with me.
Do you also actively go out of your way to recommend your friends and family cheat in general?