This seems too complacent to me. Any long-lived social media or communications utility should have some data retention policies which reduce the blast radius of an exploit and turn them into less of an endlessly growing radioactive waste dump of PII. I think this is especially true given how many people on LW have gone on to important positions or roles later in life (including in, say, cryptocurrency − 100% sufficient justification for meaningful hacking efforts); and remember the West Anglia or Hillary or Epstein emails, how badly even the most innocent communication could be abused by fanatics or fools or fraudsters? (I’ve been struck by how many of the ‘Epstein emails’ doing huge numbers on social media aren’t even real, and legitimated solely by the fact of a leak. In the postmodern oral culture, who bothers to factcheck anything, or so much as include a URL?)
Given how serious Mythos seems to be, and that information leaks are irreversible and the fact that it’s only going to escalate (remember, there’s usually a <=1 year lag from the best proprietary to opensource, so we may not even have until 2027 before mass attacks with zero guard rails or potential observability), it seems to me like this is a good time to implement some maximum retention period for DMs, and purge all old DMs. I would suggest something like, announce via email to people with any DMs that all pre-2026 DMs will be deleted within one month, and attach an export, and that forthgoing, all DMs will be deleted after 1 years of inactivity.
(Airgapped LW2 backups should go without saying and already exist!)
I haven’t thought about the tradeoffs here that much, but I would be very sad if I was a user on LessWrong, who forgot about the site for 1-2 years, and I come back expecting to find all my old DMs but instead they are all deleted. I expect all online services to use to keep my data and to not delete it, and I actively avoid any that don’t do that. I do not want to be in the habit of taking my own backups of all services I use.
and I come back expecting to find all my old DMs but instead they are all deleted.
That is why I said “and attach an export”.*
And personally I would rather a website delete my DMs than release them to the world. This is probably true of most of the people my DMs are with (whose opinion also matters).
* my reasoning here is that if old DMs have to live anywhere besides airgapped physically-secured encrypted backups, highly dispersed email accounts are the safest place because the main email providers are, in general, vastly more secure than LW2 is, and better equipped to respond rapidly to hacks, as well as extensive controls to limit exfiltration; they all have early access to Mythos-class models to reduce damage early; and they are ‘too big to fail’ in the sense that if something like Gmail is cracked wide open and leaked, it will likely be such a global cataclysm that people really won’t be able to abuse LW-related parts especially badly.
Ooh, interesting, I did fail to properly parse that you suggested directly attaching a DM export to the email. Yeah, that makes this less costly, though IMO still too annoying for anything I would want to use (of course I would prefer this over my DMs getting broadcast to the world, but really in almost any future I can see, the probability of anything like that still stays below 5%).
Why do you think it is below 5%? LW2 is already a viable hacking target just for obscure reasons like ‘stealing LLM API keys to power further hacking or exploitation’ - which we know because did that not already happen? Then there’s the cryptocurrency or political activism or blackmail angles. Do you just expect to be able to patch LW2 faster than attacker capabilities will scale?
To me, it seems like the obvious world we are headed for is one where Mythos+ level autonomous hacking capabilities will be pervasive and ambient, and just taken for granted, in the same way that we now take for granted extensive deepfakes and LLM spam everywhere, like portscanning or automated exploit suites of blogs or tailored phishes for high-value individuals, or...
No, the thing that seems unlikely is someone hacking us and then broadcasting your DMs to the world. As Robert says in the OP, attacks where someone uses any credentials or crypto-wallet passwords or API keys you sent in your DMs seem more likely than that, but I don’t think attackers would try to hack LessWrong to publish all the DMs. It’s not that juicy, it’s still pretty legally risky, and I expect things to scale more than that.
You could keep any export that nobody downloaded in the airgapped archives, against some future day when you find a better point on the tradeoff curve.
Not sure I understand you. So for example the US public seems pretty desensitized to the US executive having constant scandals and being blatantly corrupt, and that’s basically one big important actor doing lots of clearly really bad stuff. If there’s a constant flood of private communications being leaked I’d guess people would get really desensitized to it (as well as any particular leak being drowned out by the rest of the flood). So it would get less useful as a weapon because the public wouldn’t give as much of a hoot, is what I was trying to say.
Example that maps better: people probably care less about things said by someone that aged poorly, since there’s been a huge flood of such things due to social media.
This seems too complacent to me. Any long-lived social media or communications utility should have some data retention policies which reduce the blast radius of an exploit and turn them into less of an endlessly growing radioactive waste dump of PII. I think this is especially true given how many people on LW have gone on to important positions or roles later in life (including in, say, cryptocurrency − 100% sufficient justification for meaningful hacking efforts); and remember the West Anglia or Hillary or Epstein emails, how badly even the most innocent communication could be abused by fanatics or fools or fraudsters? (I’ve been struck by how many of the ‘Epstein emails’ doing huge numbers on social media aren’t even real, and legitimated solely by the fact of a leak. In the postmodern oral culture, who bothers to factcheck anything, or so much as include a URL?)
Given how serious Mythos seems to be, and that information leaks are irreversible and the fact that it’s only going to escalate (remember, there’s usually a <=1 year lag from the best proprietary to opensource, so we may not even have until 2027 before mass attacks with zero guard rails or potential observability), it seems to me like this is a good time to implement some maximum retention period for DMs, and purge all old DMs. I would suggest something like, announce via email to people with any DMs that all pre-2026 DMs will be deleted within one month, and attach an export, and that forthgoing, all DMs will be deleted after 1 years of inactivity.
(Airgapped LW2 backups should go without saying and already exist!)
I haven’t thought about the tradeoffs here that much, but I would be very sad if I was a user on LessWrong, who forgot about the site for 1-2 years, and I come back expecting to find all my old DMs but instead they are all deleted. I expect all online services to use to keep my data and to not delete it, and I actively avoid any that don’t do that. I do not want to be in the habit of taking my own backups of all services I use.
That is why I said “and attach an export”.*
And personally I would rather a website delete my DMs than release them to the world. This is probably true of most of the people my DMs are with (whose opinion also matters).
* my reasoning here is that if old DMs have to live anywhere besides airgapped physically-secured encrypted backups, highly dispersed email accounts are the safest place because the main email providers are, in general, vastly more secure than LW2 is, and better equipped to respond rapidly to hacks, as well as extensive controls to limit exfiltration; they all have early access to Mythos-class models to reduce damage early; and they are ‘too big to fail’ in the sense that if something like Gmail is cracked wide open and leaked, it will likely be such a global cataclysm that people really won’t be able to abuse LW-related parts especially badly.
Ooh, interesting, I did fail to properly parse that you suggested directly attaching a DM export to the email. Yeah, that makes this less costly, though IMO still too annoying for anything I would want to use (of course I would prefer this over my DMs getting broadcast to the world, but really in almost any future I can see, the probability of anything like that still stays below 5%).
Compromise: if various other platforms experience the equivalent of DMs leaking (or you otherwise update that it’s >>5%), quickly do the gwern plan?
Something like that seems pretty reasonable.
Why do you think it is below 5%? LW2 is already a viable hacking target just for obscure reasons like ‘stealing LLM API keys to power further hacking or exploitation’ - which we know because did that not already happen? Then there’s the cryptocurrency or political activism or blackmail angles. Do you just expect to be able to patch LW2 faster than attacker capabilities will scale?
To me, it seems like the obvious world we are headed for is one where Mythos+ level autonomous hacking capabilities will be pervasive and ambient, and just taken for granted, in the same way that we now take for granted extensive deepfakes and LLM spam everywhere, like portscanning or automated exploit suites of blogs or tailored phishes for high-value individuals, or...
No, the thing that seems unlikely is someone hacking us and then broadcasting your DMs to the world. As Robert says in the OP, attacks where someone uses any credentials or crypto-wallet passwords or API keys you sent in your DMs seem more likely than that, but I don’t think attackers would try to hack LessWrong to publish all the DMs. It’s not that juicy, it’s still pretty legally risky, and I expect things to scale more than that.
You could keep any export that nobody downloaded in the airgapped archives, against some future day when you find a better point on the tradeoff curve.
There’s going to be so much of this over the coming years that I’m guessing people will be desensitized and stop giving a hoot.
That is a misunderstanding of how it works. They won’t ‘stop giving a hoot’ because it remains a useful weapon.
Not sure I understand you. So for example the US public seems pretty desensitized to the US executive having constant scandals and being blatantly corrupt, and that’s basically one big important actor doing lots of clearly really bad stuff. If there’s a constant flood of private communications being leaked I’d guess people would get really desensitized to it (as well as any particular leak being drowned out by the rest of the flood). So it would get less useful as a weapon because the public wouldn’t give as much of a hoot, is what I was trying to say.
Also noticing I shifted the goalposts here from “stop giving a hoot” to “wouldn’t give as much of a hoot”. I concede the original point as worded.
Example that maps better: people probably care less about things said by someone that aged poorly, since there’s been a huge flood of such things due to social media.