This is a request for cybersecurity advice. I motivate why Lesswrong would benefit from a repository of such advice and why that is relevant for alignment projects (I’m working on one) and propose a structure for answers.
In Six Dimensions of Operational Adequacy in AGI Projects, Yudkowsky asks for
Strong opsec: Operational security adequate to prevent the proliferation of code (or other information sufficient to recreate code within e.g. 1 year) due to e.g. Russian intelligence agencies grabbing the code.
And defines the following levels:
Token: Random people are not allowed to wander through the building.
Improving: Your little brother cannot steal the IP. Stuff is encrypted. Siloed project members sign NDAs.
Adequate: Major governments cannot silently and unnoticeably steal the IP without a nonroutine effort. All project members undergo government-security-clearance-style screening. AGI code is not running on AWS, but in an airgapped server room. There are cleared security guards in the server room.
Excellent: Military-grade or national-security-grade security. (It’s hard to see how attempts to get this could avoid being counterproductive, considering the difficulty of obtaining trustworthy command and common good commitment with respect to any entity that can deploy such force, and the effect that trying would have on general mindsets.)
Security is also one of the Asilomar AI Principles:
6) Safety: AI systems should be safe and secure throughout their operational lifetime, and verifiably so where applicable and feasible.
I am a member of the AI Alignment project aintelope and work to properly secure the project as we grow. Right now, we have security at the token level (as defined above): A private Github project, a private Slack group, and some token vetting of people with access.
How can we do better? What resources are there? First, I looked at Lesswrong. There are the tags Computer Security & Cryptography and Security Mindset, which have some relevant posts, including Security Mindset and Ordinary Paranoia, and Secure homes for digital people. There is also Work on Security Instead of Friendliness? which is about the relation between AI Alignment and Security. But there is little material that helps me secure our project. I think this community, with many people working on alignment projects, needs to have better resources for security.
Please post best practice resources as answers in the below format. Select sources that you think have high quality. Use regular comments for discussion. Please upvote if you agree with the assessment based on quality. If you disagree with the level, use the factual agreement up/down.
Level: Token, Improving, Adequate, or Excellent
Linked Title (The title of the resource with a link to the source; for books, include the ISBN; consider linking to a review)
Description: A short description; may be copied from a review
Alternative Material (optional, but bonus points if you provide these): Two alternative sources that you are familiar with that are not as good as your preferred source. This is in the spirit of the rules from The Best Textbooks on Every Subject.
Start at the meta level—WHY do you care about these aspects of security, and how much are you willing to spend on it (in terms of money, effort, and interference with your actual mission)? This will probably change over time—you likely don’t have anything worth stealing or sabotaging early in a project, and only have a small number of people with reasonable trust levels. As you grow, your value as a target increases, as do the number of people you don’t personally know. Figure out what the triggers or Schelling lines are where you will make security a serious focus, rather than a nice-to-have.
Then ignore Eliezer’s levels—he’s absolutely right that it’s important, but his maturity-model approach is insufficient. Security is adversarial, which means it’s a pathological distribution of problems, not a smooth surface to approach generally. Instead, focus on Threat Modeling (https://owasp.org/www-community/Threat_Modeling is one resource), and the matching of hassle/expense of protection for yourself vs the risk/cost of loss and ease/likelihood of attack for the resources you’re protecting.
For most secrets, technological protection (encryption, access control/logging, penetration tests, etc.) is sufficient for casual or competitive attacks. Government “attacks” like subpoenas are really only preventable by simply not having what they want (delete your logs if this is your worry) or being very careful not to be in the jurisdictions with those risks. Targetted covert state-level or very organized criminal attacks are probably impossible to prevent. If an org can kidnap 51% of the leadership and torture them for their passwords, they’re in, no matter what. So probably easier to not try to secure against that, or to have anything likely to be attacked that way.
(after writing this I noticed that you’re asking for resources, not advice...)
How good are you at system admin stuff? A massive gaping potential issue is that you’re using third party services for both hosting your code (which could potentially be extracted via newer versions of Copilot, not to mention the thousands of GitHub employees) and your discussions (how much do you trust Discord to secure your data? How afraid are you of man in the middle attacks?). Both are free services, which means you’re the product.
I have no idea as to whether they are better, but here is a random list of alternatives to Discord. Or you could just go with Signal, which seems more trustworthy, at the cost of convenience.
Hosting your own git server gets rid of the issue of Microsoft accessing your code, at the cost of a lot more bother with setting things up and ensuring you have a secure server. Though I’m guessing a simple OpenBSD + git setup would totally suffice, assuming that you only use that machine for git. This would limit the git attack vector to SSH/OpenBSD—both of which would require a lot of work, at least at non NSA levels.
The above would get you to improving levels when it comes to transmission of the data. Storage is another issue. I see there are some encryption projects for git, but I don’t know anything about them. You could assume that remote access of the git server to be unlikely and leave it at that, but it’s worth considering physical access, though a secure box with a lock would help a lot with that. Still not adequate, though. Adequate at a minimum would require you physically working from the same location and having discussions in person to minimize the risks of overhearing. Though it adds a nice juicy target for bugging, etc.
How do you run code? If it’s on a remote server, then you’re back to trusting whoever is hosting it. If you do it locally, then you have to trust yourself (or your distro) to setup the appropriate security. Are you more worried about something getting in, or something getting out?
This is asking more questions than giving specific advice on which specific practices are appropriate for (one of) the four levels. I will post an example of specific advice to get this started.
(this is intended as an example of the template to use and not as serious advice—I googled some results and made it up on the spot)
Level: Token
How to Secure a Website from Hackers [13-Step Guide]
Description:
Describes specific steps to secure a website (e.g., blog or e-commerce webshop) that runs as an installed application on a server that you control. Requires an understanding of web applications, internet basics (URLs), possibly installing applications on Linux and running Linux commands.
Alternatives: I googled “how to secure a webshop” and looked at the first hits. The above one was the best. Here are two others that are simpler:
How to Secure a Website in 2022 | 7 Tips to Keep It Secure
E-commerce Website Security: 5 Best Practices to Protect Your Online Store