Hmm; your second point is pretty compelling to me (“when I take human-written text and ask AI to translate it, it does not get flagged as AI-generated”). I’m curious how robust that is.
I don’t find your argument about probably-Italian-first compelling yet: it seems plausible/likely to me that various folks contributed to the writing who are e.g. Pope Leo himself, or others high up in the Catholic church (perhaps selecting partly for tech-familiarity), and English fluency seems as likely there as Italian fluency to me. I would guess folks who grew up in (Rome, especially Rome outside the Vatican) are more Italian-fluent, but don’t see why I should expect authorship-average to be folks native to Rome. Maybe I’m being dense.
Hmm; your second point is pretty compelling to me (“when I take human-written text and ask AI to translate it, it does not get flagged as AI-generated”). I’m curious how robust that is.
Literally 100% for me asking Claude to take text and translate it (including old Papal encyclicals), but that’s small n and maybe it is copying the style of publicly-available human-written translations. That said Pangram gives you a couple of free checks a day and I’d encourage people to play around with this!
I’ve just tried this once, getting Claude to write a paragraph in Latin, translating it to English myself, and plugging it into Pangram. Surprisingly to me, Pangram flags it as 100% AI-generated, even though it doesn’t sound all that AI-ish to my ear (but admittedly I don’t have the most acute ear for such things). Also note that the translation was kind of a rush, and I could probably have made it sound more pleasing (e.g. “a big rock” could probably have been “a large rock”, and “I’m in the habit of walking” is not obviously the best way of rendering “ambulare soleo”).
I agree that the authors are probably not native to Rome. That said, my guess is most Vatican officials are either literally Italian (recall that John Paul II was the first non-Italian pope in a long time) or from a bunch of random countries such that the natural language to work in is that of the country that you’re in (which in this case is basically Italian).
I also agree these arguments aren’t air-tight, and would like to hear from people who know more (e.g. Vatican reporters).
Hmm, Daniel Gallagher, a strong Latinist who’s worked in the Curia, writes:
Latin is still the “official” language of the Roman Catholic Church and the Magisterium, and I have little doubt anything will ever take its place. But it is not the “working” language of the Roman Curia, and likely never will be again. Italian has had that status for some time, though English has practically replaced it over the last two decades. A few Cardinals have mentioned the importance of having had English-speaking electors on hand during the recent conclave, as many members of the College who needed help did not have Italian as a second language.
(h/t Linch Zhang for finding this article)
I think I’m now falling back to “if it wasn’t composed in Italian, I’m confused why the Italian version would show as more AI-generated”.
From the same piece, this paragraph makes it sound like Italian is the working language of encyclicals (albeit not definitively). Bolding mine:
Whereas a hundred years ago the Latin version was produced first and then the vernaculars were rendered as translations of it, now the Latin version is the last to be completed and is essentially a fine-tuned and carefully raked Latin translation of the Italian.
Hmm; your second point is pretty compelling to me (“when I take human-written text and ask AI to translate it, it does not get flagged as AI-generated”). I’m curious how robust that is.
I don’t find your argument about probably-Italian-first compelling yet: it seems plausible/likely to me that various folks contributed to the writing who are e.g. Pope Leo himself, or others high up in the Catholic church (perhaps selecting partly for tech-familiarity), and English fluency seems as likely there as Italian fluency to me. I would guess folks who grew up in (Rome, especially Rome outside the Vatican) are more Italian-fluent, but don’t see why I should expect authorship-average to be folks native to Rome. Maybe I’m being dense.
I have 3 lines of arguments against the translation artifact hypothesis (though my post takes the English-language article as a starting point and doesn’t take a strong position on the source language) https://linch.substack.com/i/199316613/probably-not-a-translation-artifact
Literally 100% for me asking Claude to take text and translate it (including old Papal encyclicals), but that’s small n and maybe it is copying the style of publicly-available human-written translations. That said Pangram gives you a couple of free checks a day and I’d encourage people to play around with this!
(Ideally other people would check with LLMs other than Claude)
The converse question is more of a bother, but: any data on how human translations of AI-generated text show up on Pangram?
I’ve just tried this once, getting Claude to write a paragraph in Latin, translating it to English myself, and plugging it into Pangram. Surprisingly to me, Pangram flags it as 100% AI-generated, even though it doesn’t sound all that AI-ish to my ear (but admittedly I don’t have the most acute ear for such things). Also note that the translation was kind of a rush, and I could probably have made it sound more pleasing (e.g. “a big rock” could probably have been “a large rock”, and “I’m in the habit of walking” is not obviously the best way of rendering “ambulare soleo”).
None that I have.
I agree that the authors are probably not native to Rome. That said, my guess is most Vatican officials are either literally Italian (recall that John Paul II was the first non-Italian pope in a long time) or from a bunch of random countries such that the natural language to work in is that of the country that you’re in (which in this case is basically Italian).
I also agree these arguments aren’t air-tight, and would like to hear from people who know more (e.g. Vatican reporters).
ChatGPT5.5 estimates 50% Italian-first, 35% English-first, as prior.
Academia often has people “from a bunch of random countries” and often defaults to English.
Hmm, Daniel Gallagher, a strong Latinist who’s worked in the Curia, writes:
(h/t Linch Zhang for finding this article)
I think I’m now falling back to “if it wasn’t composed in Italian, I’m confused why the Italian version would show as more AI-generated”.
From the same piece, this paragraph makes it sound like Italian is the working language of encyclicals (albeit not definitively). Bolding mine: