I actually think we’re on the same page about a lot of what you’re saying.
But those cases add up and can have impact over time.
I agree with this. It is valuable for me to write a guide on those cases too. If I have to write only one guide I’ll write the one I’m writing now. But yes both are valuable.
No, I haven’t gone through and systematically analyzed everything I could find in a single process. But I don’t think you have either. How did you identify the cases you thought about? Just off the top of my head, where’s Mark Klein[1]? Where’s Deep Throat[2]? Did you use a systematic and relatively unbiased method of finding the “data” you’re relying on?
I have some notes on both Mark Klein and Deep Throat, haven’t published. I do plan to publish a proper database of atleast the top 20 or so leaks (and ideally a lot more) with fact-checked information.
Short version is Mark Klein did not release classified information, and Mark Felt (Deep Throat) operated over 50 years ago in a different technological and legal environment. The NSA did not have same level of technical capabilities, nor had it bureaucratically hacked the FBI back then.
I don’t actually think it’s completely futile to try to do this stuff with lasting anonymity, whereas I think you may believe that it usually is. But to do that, you have to be a certain kind of person, with a certain kind of knowledge, and a certain mindset. If you’re not that person, you may have to become that person first. Which, by the way, can make you suspicious in itself.
On the other hand, doing it openly also requires being a certain kind of person. Instead of the ability to do OPSEC, you need the ability to handle blowing up your life, and possibly the lives of people who depend on you. Also not easy to become.
The main reason I’m pessimistic about anonymity in this case is the extent to which the NSA tracks who downloads every document. The list of suspects is small enough that everyone can be investigated, which means your best bet might be to attempt to falsely incriminate someone else in that pool. Apart from that having questionable morals, it’s also really hard to do for someone who lacks experience in this type of work.
If you are giving up on anonymity, I agree you have to be able to handle blowing up your life in many respects. I’ve recently been reading up on the possibility of taking spouse or children with you when you whistleblow. I do think this guide needs more resources on mental health.
If you give people counsel about risks, you need to make them understand those risks, and exactly how and how much your guidance changes those risks. Short guides can’t even do that much, let alone do much beyond the most obvious things to mitigate those risks.
It might actually be safer to force people to think for themselves. Even the sort of OPSEC that keeps you from getting caught before you leak takes a certain mindset.
This is valuable input, thanks for sharing this. I do understand opsec is a mindset not just a set of rules.
Someone who has the mindset still needs a clear set of rules though. And I think it’s better for a team of experts from the outside to write that set of rules, instead of the person themselves. They can make an attempt to adapt the rules to their circumstances.
I think it would be extremely valuable if there were fast ways to transmit this mindset to someone who does not already have the mindset. I’d love more input on how to do this. I will also think more about it myself. Most people at OpenAI/Deepmind/Anthropic are software developers and hence teaching them the mindset can be done more quickly, assuming they dont have it already.
They create false expecations, giving people the idea that they know what they need to do, without equipping them to understand the limitations of the approach, or notice when it need to change to match individual, unforeseeable circumstances.
I think the ethics of this come down to ultimately what the probability of success is. I think if you’re going the Snowden route of fleeing the country, you can afford to make some opsec mistakes and still escape with your life. The bar is nowhere as high as it would be, if I were writing a guide for someone trying to stay anonymous within the US. My guide explicitly says you will be doxxed eventually, it’s just a question of how soon.
There’s usually a manual. If they can’t understand the manual once they’re told they need to, they won’t understand what I say, either.
I have seen some of these manuals on dark web. You can send me anonymous email at samuel.da.shadrach@gmail.com if you want to send me any. (I don’t operate an anonymous email myself for this circumstance, I think it could give me false sense of security if I create one and then post it here under my real name.)
The main reason I don’t want to tell people to just refer to a manual is none of the manuals are tailor made to this circumstance, they’re usually written by someone with a different circumstance in mind, such as a drug vendor or cyberhacker or whatever.
And a whistleblower has limited time and energy, they can’t go through 10 different manuals and carefully pick and choose what applies to their situation. They need a clear set of rules ready-to-go IMO.
It’d also have to be maintained, or it’d have dangerous inaccuracies scarily soon. Things change all the time.
Agree. My platonic ideal is atleast once a month, all the experts involved in writing the guide go through and approve any changes to it.
Short version is Mark Klein did not release classified information,
Right. And I think at least part of the reason he got away with it was that particular people (and courts) at the time felt constrained by the rule of law. This may not always apply.
The NSA did not have same level of technical capabilities, nor had it bureaucratically hacked the FBI back then.
I’m going to respond to this at more length, but I’ve moved most of it down because I think it’s a lower priority. The main point I want to make is that there’s a misleading attitude out there about the NSA, and letting your thoughts be all “NSA” may not be healthy.
I agree that everybody who’s watching will cooperate on something like what we’re talking about here.
The list of suspects is small enough that everyone can be investigated,
That depends on circumstances, though. Sometimes yes, sometimes no. Some things you might want to leak may have fairly wide internal distribution. And sometimes there’s a way to get something outside of the normal channels. And sometimes they really don’t want to risk going after the wrong person, so those investigations need to be more airtight.
But surely not always.
which means your best bet might be to attempt to falsely incriminate someone else in that pool. Apart from that having questionable morals, it’s also really hard to do for someone who lacks experience in this type of work.
Lots of people get caught trying to do that.
Someone who has the mindset still needs a clear set of rules though. And I think it’s better for a team of experts from the outside to write that set of rules, instead of the person themselves. They can make an attempt to adapt the rules to their circumstances.
Well, what kind of rules do you want?
That litany of questions I posted could be considered as a checklist. But I’m not sure that asking them equips you to recognize when you have or haven’t actually answered them.
I might add something about thinking about the other side’s constraints and available moves, and about not being comfortable that you understand something until you’ve confirmed how it works at the “gears level”. That’s where all the factual knowledge comes in.
I think it would be extremely valuable if there were fast ways to transmit this mindset to someone who does not already have the mindset.
Other than stuff like those questions, I don’t know a way. I think it’s partly about seeing these sorts of games played a lot of ways, and partly about the innate ability to think through all the possibilities with the right kind of paranoia. Paranoia that’s skeptical of itself.
In some sense it’s like the skill of finding all the cases in a mathematical proof, except that in practice you also have to be able to decide you’ll live with a “counterexample” if it’s improbable enough. Which, of course, means judging how improbable it actually is. It’s easy to fall into error there, in either direction.
… and of course this is all “theoretical” on my part in that I’ve never actually done it when anybody was really going to go after me.
I think the ethics of this come down to ultimately what the probability of success is.
… and to whether the person at risk knows that probability...
Most of that stuff is drivel. It tends to be a total mishmash of truths, half-truths, unsupported folklore, misunderstandings, outright lies, and regurgitated disinformation, collected by people with no coherent internal model of How Things Work to measure any of it against. Then it gets filtered through whatever conspiracy theories the writers use to feed their addiction to feeling like they know the Hidden Truth(TM), with incompatible evidence discarded.
Trying to dig real information out of that material is at best horribly frustrating, and it’s terrible for helping you get a coherent model, especially a coherent accurate model. At most it might give you some search terms, and even then you need to be super skeptical of what you find.
And not any “howtos”.
I mean the actual manuals that actually explain how the systems involved work.
For technical information, that means not something entitled “How to HaxOr on the Dark Web lol”, but the manual for the OS. The manuals for servers and monitoring systems. The protocol specifications. Security product literature. Technical security standards. Architecture textbooks. There’s no royal road; you have to actually understand how things work.
Figuring out how organizations and institutions work is harder than technology, because they vary, they have fewer relevant physical or mathematical constraints, they don’t tend to have public documentation, and they don’t always follow their own internal “rules”. You can piece some of it together, and for the rest at least you can often figure out what you don’t know. You can look at court documents, official procedures, security management standards, even amusing personal anecdotes about bureaucracy. You can infer some of it from the features people have seen fit to put into technology that supports the organizations. But you won’t get it from something called the “The Truth about the NSA(TM)”. If anything there are more silly conspiracy theories about institutions than about technology.
And a whistleblower has limited time and energy, they can’t go through 10 different manuals and carefully pick and choose what applies to their situation. They need a clear set of rules ready-to-go IMO.
This may be the crux of the issue. I would tend to say that if you’re not already pretty sophisticated, maybe you shouldn’t do it. Not just because you don’t know your risks, not just because you may be unprepared either to stay anonymous or to deal with the consequences of not being… but also because you may not understand the actual impact, or lack thereof, that your disclosure will have when it hits the real world.
Agree. My platonic ideal is atleast once a month, all the experts involved in writing the guide go through and approve any changes to it.
Sure, but it’s super hard to actually do even a far less perfect version of that.
Surveillance and “the NSA”
A couple of reordered quotes--
The main reason I’m pessimistic about anonymity in this case is the extent to which the NSA tracks who downloads every document.
I think we’ll probably agree that, at least in your main scenario, it doesn’t actually matter much exactly who watches what. You can expect them all to be cooperating, and it may not matter much in the end. But I think it may matter enough that it pays to keep a clear model of them. And for the sake of that...
It’s not just the NSA spying on everything, nor the NSA and the FBI, nor any fixed set of people. Also, the NSA qua NSA isn’t as omniscient as some people think. And information sharing isn’t about “hacking”, even bureaucratic hacking. If you think of all the people who are watching in terms of a monolithic, all-knowing “NSA”, you may make mistakes.
Basically every modern institution, even non-secretive ones, logs internal downloads (and external network connections, and more). You can end up doing that without even trying, just because of the software defaults. I believe the NSA does not usually have access to such internal logs. But, again, for this case, they’re on the same side as the people who do.
The people who matter most in terms of noticing your initial acquisition of whatever information will be your organization’s internal security and counterintelligence functions.
Where you get into trouble with the NSA is out on the public Internet. After you exfiltrate your data, the NSA is the agency that may be able to figure out where they went, or link your activities outside of the organization you’re whistleblowing on with your activities inside of it.
The [Watergate—jbash] NSA did not have same level of technical capabilities, nor had it bureaucratically hacked the FBI back then.
I think the technological change is really the key point. Back then, you couldn’t keep a log of every download; there were no downloads. If somebody photocopied something, that would at most show up on a total copies counter on one of any number of machines. And there weren’t likely to be permanently recorded cameras in that parking garage (which would mostly likely belong to the garage, not the NSA).
As for organizations, the FBI at the time was the J. Edgar Hoover FBI. I’m not sure anybody had to hack it to weaponize it. And the NSA was still the totally secret No Such Agency, and was almost entirely focused on foreign COMINT (it’s still basically about COMINT, and theoretically foreign).
I’m not sure what “bureaucratically hacked” means. I don’t believe the NSA has substantially infiltrated the FBI in some undercover way. A lot of interagency barriers went away with the (stupid) USA-PATRIOT act and the surrounding restructurings. But those changes weren’t “hacking”. They were openly imposed by official policy from above and outside both agencies (even though they were probably welcomed on both sides).
And I think at least part of the reason he got away with it was that particular people (and courts) at the time felt constrained by the rule of law. This may not always apply.
There is a more recent list of examples of people who also got away with publicly leaking summaries but not leaking classified information. Your point is also true, but I think classified information being leaked is a bigger factor. It’ll be easier for me to argue all this once I publish an actual list of case studies.
Sorry, I need more time for that.
For technical information, that means not something entitled “How to HaxOr on the Dark Web lol”, but the manual for the OS. The manuals for servers and monitoring systems. The protocol specifications. Security product literature. Technical security standards. Architecture textbooks. There’s no royal road; you have to actually understand how things work.
Yes I agree this is required for the multiple books opsec guide, not the 10-page quick guide. Again, the question comes back to probabilities. These are very much not the actual numbers but if P(no guide, no prison) = 50%, P(10-page guide, no prison) = 65%, P(1000-page guide, no prison) = 80%, then it is worth publishing a 10-page guide.
I would tend to say that if you’re not already pretty sophisticated, maybe you shouldn’t do it. Not just because you don’t know your risks, not just because you may be unprepared either to stay anonymous or to deal with the consequences of not being… but also because you may not understand the actual impact, or lack thereof, that your disclosure will have when it hits the real world.
You can have wide uncertainty on every single one of these questions and still conclude that whistleblowing is the correct decision. See the example probabilities above.
I agree that my guide should include a section on predicting potential outcomes after you whistleblow. I will definitely do this. Thank you for the suggestion.
Basically every modern institution, even non-secretive ones, logs internal downloads (and external network connections, and more). You can end up doing that without even trying, just because of the software defaults. I believe the NSA does not usually have access to such internal logs. But, again, for this case, they’re on the same side as the people who do.
The people who matter most in terms of noticing your initial acquisition of whatever information will be your organization’s internal security and counterintelligence functions.
Where you get into trouble with the NSA is out on the public Internet. After you exfiltrate your data, the NSA is the agency that may be able to figure out where they went, or link your activities outside of the organization you’re whistleblowing on with your activities inside of it.
I agree this model is valuable to have, and I appreciate you writing this up.
I honestly do think though that by 2027, there is a good chance internal security for AI companies will directly report to the head of the NSA and ultimately the president, not to the head of labs.
And not just in a legal abstract sense like how Lockheed Martin’s various security teams may be accountable to the govt, but in the sense of how Manhattan project security directly reported to US military generals who frequently visited the sites for inspections.
I would not be surprised if NSA leaders built war rooms and private residences inside the datacenter compound, although this might be me speculating too far. If it happens though it may take longer than 2027, maybe 2029? It depends on timelines honestly.
Other than stuff like those questions, I don’t know a way. I think it’s partly about seeing these sorts of games played a lot of ways, and partly about the innate ability to think through all the possibilities with the right kind of paranoia. Paranoia that’s skeptical of itself.
I’ll try to think of a way though. It seems valuable to do.
I think the technological change is really the key point
Agree!
They were openly imposed by official policy from above and outside both agencies (even though they were probably welcomed on both sides).
Snowden in Permanent Record considers Dick Cheney, vice president and Michael Hayden, NSA director co-conspirators. I don’t have a lot of data to argue this but I do think the FBI director was intentionally kept out of the matter.
Your description is not wrong, there’s nuances here I haven’t tried to fully understand yet either. Thanks for the pointer that this may be worth doing.
I love your reply.
I actually think we’re on the same page about a lot of what you’re saying.
I agree with this. It is valuable for me to write a guide on those cases too. If I have to write only one guide I’ll write the one I’m writing now. But yes both are valuable.
I have some notes on both Mark Klein and Deep Throat, haven’t published. I do plan to publish a proper database of atleast the top 20 or so leaks (and ideally a lot more) with fact-checked information.
Short version is Mark Klein did not release classified information, and Mark Felt (Deep Throat) operated over 50 years ago in a different technological and legal environment. The NSA did not have same level of technical capabilities, nor had it bureaucratically hacked the FBI back then.
The main reason I’m pessimistic about anonymity in this case is the extent to which the NSA tracks who downloads every document. The list of suspects is small enough that everyone can be investigated, which means your best bet might be to attempt to falsely incriminate someone else in that pool. Apart from that having questionable morals, it’s also really hard to do for someone who lacks experience in this type of work.
If you are giving up on anonymity, I agree you have to be able to handle blowing up your life in many respects. I’ve recently been reading up on the possibility of taking spouse or children with you when you whistleblow. I do think this guide needs more resources on mental health.
This is valuable input, thanks for sharing this. I do understand opsec is a mindset not just a set of rules.
Someone who has the mindset still needs a clear set of rules though. And I think it’s better for a team of experts from the outside to write that set of rules, instead of the person themselves. They can make an attempt to adapt the rules to their circumstances.
I think it would be extremely valuable if there were fast ways to transmit this mindset to someone who does not already have the mindset. I’d love more input on how to do this. I will also think more about it myself. Most people at OpenAI/Deepmind/Anthropic are software developers and hence teaching them the mindset can be done more quickly, assuming they dont have it already.
I think the ethics of this come down to ultimately what the probability of success is. I think if you’re going the Snowden route of fleeing the country, you can afford to make some opsec mistakes and still escape with your life. The bar is nowhere as high as it would be, if I were writing a guide for someone trying to stay anonymous within the US. My guide explicitly says you will be doxxed eventually, it’s just a question of how soon.
I have seen some of these manuals on dark web. You can send me anonymous email at samuel.da.shadrach@gmail.com if you want to send me any. (I don’t operate an anonymous email myself for this circumstance, I think it could give me false sense of security if I create one and then post it here under my real name.)
The main reason I don’t want to tell people to just refer to a manual is none of the manuals are tailor made to this circumstance, they’re usually written by someone with a different circumstance in mind, such as a drug vendor or cyberhacker or whatever.
And a whistleblower has limited time and energy, they can’t go through 10 different manuals and carefully pick and choose what applies to their situation. They need a clear set of rules ready-to-go IMO.
Agree. My platonic ideal is atleast once a month, all the experts involved in writing the guide go through and approve any changes to it.
Right. And I think at least part of the reason he got away with it was that particular people (and courts) at the time felt constrained by the rule of law. This may not always apply.
I’m going to respond to this at more length, but I’ve moved most of it down because I think it’s a lower priority. The main point I want to make is that there’s a misleading attitude out there about the NSA, and letting your thoughts be all “NSA” may not be healthy.
I agree that everybody who’s watching will cooperate on something like what we’re talking about here.
That depends on circumstances, though. Sometimes yes, sometimes no. Some things you might want to leak may have fairly wide internal distribution. And sometimes there’s a way to get something outside of the normal channels. And sometimes they really don’t want to risk going after the wrong person, so those investigations need to be more airtight.
But surely not always.
Lots of people get caught trying to do that.
Well, what kind of rules do you want?
That litany of questions I posted could be considered as a checklist. But I’m not sure that asking them equips you to recognize when you have or haven’t actually answered them.
I might add something about thinking about the other side’s constraints and available moves, and about not being comfortable that you understand something until you’ve confirmed how it works at the “gears level”. That’s where all the factual knowledge comes in.
Other than stuff like those questions, I don’t know a way. I think it’s partly about seeing these sorts of games played a lot of ways, and partly about the innate ability to think through all the possibilities with the right kind of paranoia. Paranoia that’s skeptical of itself.
In some sense it’s like the skill of finding all the cases in a mathematical proof, except that in practice you also have to be able to decide you’ll live with a “counterexample” if it’s improbable enough. Which, of course, means judging how improbable it actually is. It’s easy to fall into error there, in either direction.
… and of course this is all “theoretical” on my part in that I’ve never actually done it when anybody was really going to go after me.
… and to whether the person at risk knows that probability...
Egad, no, not those “manuals”[1]!
Most of that stuff is drivel. It tends to be a total mishmash of truths, half-truths, unsupported folklore, misunderstandings, outright lies, and regurgitated disinformation, collected by people with no coherent internal model of How Things Work to measure any of it against. Then it gets filtered through whatever conspiracy theories the writers use to feed their addiction to feeling like they know the Hidden Truth(TM), with incompatible evidence discarded.
Trying to dig real information out of that material is at best horribly frustrating, and it’s terrible for helping you get a coherent model, especially a coherent accurate model. At most it might give you some search terms, and even then you need to be super skeptical of what you find.
And not any “howtos”.
I mean the actual manuals that actually explain how the systems involved work.
For technical information, that means not something entitled “How to HaxOr on the Dark Web lol”, but the manual for the OS. The manuals for servers and monitoring systems. The protocol specifications. Security product literature. Technical security standards. Architecture textbooks. There’s no royal road; you have to actually understand how things work.
Figuring out how organizations and institutions work is harder than technology, because they vary, they have fewer relevant physical or mathematical constraints, they don’t tend to have public documentation, and they don’t always follow their own internal “rules”. You can piece some of it together, and for the rest at least you can often figure out what you don’t know. You can look at court documents, official procedures, security management standards, even amusing personal anecdotes about bureaucracy. You can infer some of it from the features people have seen fit to put into technology that supports the organizations. But you won’t get it from something called the “The Truth about the NSA(TM)”. If anything there are more silly conspiracy theories about institutions than about technology.
This may be the crux of the issue. I would tend to say that if you’re not already pretty sophisticated, maybe you shouldn’t do it. Not just because you don’t know your risks, not just because you may be unprepared either to stay anonymous or to deal with the consequences of not being… but also because you may not understand the actual impact, or lack thereof, that your disclosure will have when it hits the real world.
Sure, but it’s super hard to actually do even a far less perfect version of that.
Surveillance and “the NSA”
A couple of reordered quotes--
I think we’ll probably agree that, at least in your main scenario, it doesn’t actually matter much exactly who watches what. You can expect them all to be cooperating, and it may not matter much in the end. But I think it may matter enough that it pays to keep a clear model of them. And for the sake of that...
It’s not just the NSA spying on everything, nor the NSA and the FBI, nor any fixed set of people. Also, the NSA qua NSA isn’t as omniscient as some people think. And information sharing isn’t about “hacking”, even bureaucratic hacking. If you think of all the people who are watching in terms of a monolithic, all-knowing “NSA”, you may make mistakes.
Basically every modern institution, even non-secretive ones, logs internal downloads (and external network connections, and more). You can end up doing that without even trying, just because of the software defaults. I believe the NSA does not usually have access to such internal logs. But, again, for this case, they’re on the same side as the people who do.
The people who matter most in terms of noticing your initial acquisition of whatever information will be your organization’s internal security and counterintelligence functions.
Where you get into trouble with the NSA is out on the public Internet. After you exfiltrate your data, the NSA is the agency that may be able to figure out where they went, or link your activities outside of the organization you’re whistleblowing on with your activities inside of it.
I think the technological change is really the key point. Back then, you couldn’t keep a log of every download; there were no downloads. If somebody photocopied something, that would at most show up on a total copies counter on one of any number of machines. And there weren’t likely to be permanently recorded cameras in that parking garage (which would mostly likely belong to the garage, not the NSA).
As for organizations, the FBI at the time was the J. Edgar Hoover FBI. I’m not sure anybody had to hack it to weaponize it. And the NSA was still the totally secret No Such Agency, and was almost entirely focused on foreign COMINT (it’s still basically about COMINT, and theoretically foreign).
I’m not sure what “bureaucratically hacked” means. I don’t believe the NSA has substantially infiltrated the FBI in some undercover way. A lot of interagency barriers went away with the (stupid) USA-PATRIOT act and the surrounding restructurings. But those changes weren’t “hacking”. They were openly imposed by official policy from above and outside both agencies (even though they were probably welcomed on both sides).
And please don’t say “Dark Web”. It romanticizes a bunch of unconnected Web sites, many of them run by fools and cranks, into something they’re not.
There is a more recent list of examples of people who also got away with publicly leaking summaries but not leaking classified information. Your point is also true, but I think classified information being leaked is a bigger factor. It’ll be easier for me to argue all this once I publish an actual list of case studies.
Sorry, I need more time for that.
Yes I agree this is required for the multiple books opsec guide, not the 10-page quick guide. Again, the question comes back to probabilities. These are very much not the actual numbers but if P(no guide, no prison) = 50%, P(10-page guide, no prison) = 65%, P(1000-page guide, no prison) = 80%, then it is worth publishing a 10-page guide.
You can have wide uncertainty on every single one of these questions and still conclude that whistleblowing is the correct decision. See the example probabilities above.
I agree that my guide should include a section on predicting potential outcomes after you whistleblow. I will definitely do this. Thank you for the suggestion.
I agree this model is valuable to have, and I appreciate you writing this up.
I honestly do think though that by 2027, there is a good chance internal security for AI companies will directly report to the head of the NSA and ultimately the president, not to the head of labs.
And not just in a legal abstract sense like how Lockheed Martin’s various security teams may be accountable to the govt, but in the sense of how Manhattan project security directly reported to US military generals who frequently visited the sites for inspections.
I would not be surprised if NSA leaders built war rooms and private residences inside the datacenter compound, although this might be me speculating too far. If it happens though it may take longer than 2027, maybe 2029? It depends on timelines honestly.
I’ll try to think of a way though. It seems valuable to do.
Agree!
Snowden in Permanent Record considers Dick Cheney, vice president and Michael Hayden, NSA director co-conspirators. I don’t have a lot of data to argue this but I do think the FBI director was intentionally kept out of the matter.
Your description is not wrong, there’s nuances here I haven’t tried to fully understand yet either. Thanks for the pointer that this may be worth doing.