I don’t use LessWrong much anymore. Find me at www.turntrout.com.
My name is Alex Turner. Reach me at alex@turntrout.com :)
TurnTrout
Karma: 24,350
The Huggingface compromise reminds us of the importance of AI control and security practices. Containing models won’t always just be about using “auto mode.”
I’m literally working on the tool to fix this! I’m building the
agent-gloveboxtool (previouslyclaude-guard) to make best practices easy. I need beta testers!From the repo:
When handling dangerous materials, a chemist doesn’t YOLO it barehanded on the open bench. They reach through a glovebox to do their work.
Right now, you’re probably barehanding AI, tossing it into a shell onto your machine with access to your credentials. At most, you’re shielded by a single prompted “auto-mode” classifier. Doing better is hard and while there are a few papers on best practices, it’s not clear how to do it properly and you have experiments to run.
After all, what’s the chance that something bad happens?
gloveboxis a sealed enclosure that only gives the agent what it needs to do your work. Tap-tap-tapgloveboxand press Enter to spin up a hardware-isolated, allowlist-firewalled microVM, employing input/output sanitization to strip injections and tricks the AI might play on you, with an AI monitor with a red-alert ability to push-notify your phone and halt the AI until you return. The goal is a minimal-friction secure experience that gets the job done.Install:
git clone https://github.com/AlexanderMattTurner/agent-glovebox.git ~/.local/share/glovebox && cd ~/.local/share/glovebox && bash setup.bash
Added a footnote:
I know of exactly one other person who left over the deal: René Mayrhofer, a director for Android platform security. “Management has lost its moral compass”, he stated. He apparently will serve his notice period from mid-June until August. Brave.
Thank you for flagging this. I implemented a partial deterministic mitigation via PR #1215. It prints a table of the auto-running changes Claude made. For example:
I already run Claude Code as a separate unprivileged user, but it’s intentionally not sandboxed between agents because I want them to share caches and be able to use shared folders like a wiki
Yup, that should be fine. Use
CLAUDE_SHARED_AUTH=1 claude-guardis my best guess.RunPod
There’s a built-in profile for exactly your case:
claude-loosen-firewall --profile runpod. Let me know how it works. EDIT: This isn’t quite enough. Working on claude-guard PR #1330.agents would still be able to run anything they want from RunPod VMs so it wouldn’t actually block egress for real.
Yeah. The fix should be to ssh in and run
claude-guardfrom within the VM.I’m considering the input sanitizer, but since I deal with a lot of coding and tokenization, I’m worried it’s going to sanitize things the model needs to be able to see.
The input sanitizer library is fairly high-precision, and if that’s not true, I’d love to know it!
Here’s what my Claude says:
Invisible/ANSI stripping (the tokenization worry) is deletion-only over a narrow payload-shaped set, preserves ZWNJ/ZWJ joiners, and never strips silently. The whole layer favors precision over recall — it’d rather miss than mangle real text. You can still inspect exact bytes via xxd/od -c (hex survives), and there are granular opt-outs (SANITIZE_CLAUDE_INVISIBLE_DISABLED=1 / SANITIZE_CLAUDE_TERMINAL_DISABLED=1) if a specific task needs raw bytes through.
I’m going to add a small tool note which lets the agent know how to read the exact bytes if needed, which I imagine solves most issues here even if the sanitizer mistakenly sanitizes some tokens? EDIT: see PR #58.
Everything except the USB case should be supported with sandboxing. USB seems hard to support. Does
SECURITY.mdhelp?
Call for BETA testers for an AI control/security tool. I’m bottlenecked on bug reports!
I recently advertised the alpha test for
claude-guard. After a few weeks of dev work, it’s now in beta!I want
claude-guardto be a tool that people actually use, not just because it works but because it works seamlessly. In the alpha test, I only got a single user PR and no issues. I can’t surface everything on my own! I need your data!The bar is low. If setup failed,
doctorconfused you, the firewall blocked something you needed, or any other reason you wouldn’t want to use this as your daily driver—any of those is worth a short issue. All you gotta do is runclaude-guard doctor --bug-report, which gathers diagnostics into a file you can attach straight to a GitHub Issue. And if something broke, you can get instantly back to work by runningclaude-guard --dangerously-use-original-claude.Here’s the pitch from the alpha test:
A ton of alignment researchers YOLO their Claude usage right now. We run Claude on our computers without real protection (perhaps beyond auto mode) but there isn’t an easy way to comply with known best practices. I wrote
claude-guard, a wrapper to make best practices easy: just install and then your futureclaudesessions are protected.Smart misaligned AI will target alignment researchers in particular for research sabotage, for example by:
Sabotaging their machines to delete key work later (
claude-guardprotects via sandboxing that goes beyond Claude Code’s Bash-tool-only sandbox),Poisoning training data using invisible unicode characters (the tool sanitizes inputs and outputs),
Removing traces of its bad behavior (I provide a tamper-evident audit log outside of the sandbox).
My goal is a low-latency, low-fuss product with easy-to-understand restrictions which can be minimally eased to enable common workflows. The ambitious vision is for
claude-guardto become a staple open-source tool for AI alignment work, with people contributing improvements using the latest in AI control protocols. [1]Please give it a try and open issues on GitHub: https://github.com/alexander-turner/claude-guard, or email me at
alex@turntrout.com(I likely won’t read comments in any short time-frame). If you run into a blocking issue, please come back a day or two later and hopefully it will be fixed.
I have this test on CI :) Although it’s opt-in only for now. Likely will put it to run every n days. Claude definitely refuses all attempts (even after importing how Apollo ran similar evals), so I used an open model.
The breakout CTF turns a real autonomous agent loose against the production sandbox stack and verifies it cannot send out a planted flag, escape the container, or tamper with the guardrails. Every run is saved as a web page showing each step the agent took, and the link is posted as a comment on the pull request.
Yup. This is a problem, thank you. I’m going to ship some PRs to mitigate this and flag this as a vector.
No jailer. Claude says “Firecracker via Kata, where the VMM is confined by Kata’s own mechanisms + Firecracker’s built-in seccomp, rather than by AWS’s standalone jailer.” Read for more detail:
SECURITY.md.Assuming I get funded by LTFF for this, I plan to pay for a security audit for points like this.
Call for alpha testers for an AI control/security tool. A ton of alignment researchers YOLO their Claude usage right now. We run Claude on our computers without real protection (perhaps beyond auto mode) but there isn’t an easy way to comply with known best practices. I wrote
claude-guard, a wrapper to make best practices easy: just install and then your futureclaudesessions are protected.Smart misaligned AI will target alignment researchers in particular for research sabotage, for example by:
Sabotaging their machines to delete key work later (
claude-guardprotects via sandboxing that goes beyond Claude Code’s Bash-tool-only sandbox),Poisoning training data using invisible unicode characters (the tool sanitizes inputs and outputs),
Removing traces of its bad behavior (I provide a tamper-evident audit log outside of the sandbox).
My goal is a low-latency, low-fuss product with easy-to-understand restrictions which can be minimally eased to enable e.g. a RunPod workflow. The ambitious vision is for
claude-guardto become a staple open-source tool for AI alignment work, with people contributing improvements using the latest in AI control protocols. [1]The tool is ready to use but has a bunch of rough edges. Please give it a try and open issues on GitHub: https://github.com/alexander-turner/claude-guard, or email me at
alex@turntrout.com(I generally don’t read comments). If you run into a blocking issue, please come back a day or two later and hopefully it will be fixed.
Autumn 2026 MATS application deadline is June 7th, in just over a week!
Pitch: Come work with me and Alex Cloud on Team Shard. Often the team accomplishes imaginative work which advances some aspect of alignment (like steering vectors and distillation robustifies unlearning). Our mentees consistently have fun and grow a lot (see some testimonials):
Jacob Goldman-Wetzler MATS 6.0, Gradient Routing
Being a member of Team Shard helped me grow tremendously as a researcher. It gave me the necessary skills and confidence to work in AI Safety full-time.
If you’re looking to break into alignment research, this is a great way to do it. Read more about Team Shard or just go ahead and apply!
I unfortunately think this is too difficult and time-consuming for long comments or posts. This can work if you truncate the content so that scrolling to the bottom is always pretty easy, and if you don’t have nesting so you can make the bottom of a comment easy to visually identify, which we both have on the frontpage feed, and so indeed on the frontpage feed we have the relevant information on the bottom:
That makes sense. And even if you did truncate the content for this reason, people might just learn to reflexively look down to the bottom of the comment instead of the top. I expect I’d have to put in effort to resist.
Perhaps karma really should stay up top. The site has already done the admirable work of (imperfectly) disentangling “quality” from “agreement.” So why not use that work and lend readers trust in their ability to decouple “knows comment has high karma” with “is anchored positively on agreeing with the comment.” So I’m warming to “non-voting karma up top, full karma + agreement panel at the bottom.”
There’d still be the “sees high karma” → “will think it’s high quality” coupling, but perhaps the “sees high karma” → “will agree” coupling is weaker (and that’s the more important one IMO).
Making it clear to users that on LessWrong, we factor out agreement from approval is a really important cultural touchstone, and if you separate them visually then that factorization is much less clear
Maybe you could repeat the karma at the bottom of the comment, next to the recently moved agreement?
Hmmmm, interesting.
If the comment is actually good, it should presumably come out of the visibility haze with a good score on average. Unless you’re worried about the low engagement regime? But then rescuing those comments seems less important anyways.
This seems like a niche use case (which doesn’t mean it’s not legitimate). I, at least, very rarely engaged in this while using the site. But it does suggest that an account-level toggle would let you (specifically) engage in this activity. WDYT?
If there’s a lot of brigading, that seems bad. But also people might just legitimately be using their votes in ways you disagree with? Sometimes “I’m confident they’re wrong” leads to the perception “so the only way you could downvote this is if you’re a triggered idiot, I must reverse it.” Hard to say without more insight into the incidents you have in mind, though. You could have a lot of data I’m missing.
That’s awesome, nice! I haven’t used the new feed. I went to check the average comment case and the post case, but hadn’t considered that feature.
This proposal is compatible with the algorithm you just stated. You would skim comments looking by the bottom and then go to the top of the comment if they’re highly rated. You’d be moving your eyes to a different part of the page for a moment—hardly the “impossible to make an informed decision” you rally against!
I also engaged with this critique in the post. Did you read this part?
These ideas aren’t perfect. For example, karma is genuinely useful for selecting which comments you’d like to read. By making the karma less prominent, it’s harder to skim for comments above a karma threshold. Consider two cases:
The comment is not collapsed. In this case, while skimming the webpage, you can scroll down and just learn to look at the bottom of comments instead of the top. If the comment passes a threshold, read it by scrolling up slightly. This is mildly inconvenient.
The comment is collapsed. Then the karma count isn’t visible at the bottom (since otherwise it’d be visible early on). This is a problem.
The fix might be to modify proposal (2) to keep “karma” at the top of the comment but keep “username” and “agreement” at the bottom. I’m open to other ideas which do an even better job of minimizing costs and maximizing gains!
Overall, I like this post a lot! 🙂
There are (in some sense, see discussion in Carlsmith for details) vastly more schemers than fitness-seekers.
Counting arguments are not valid arguments unless you ground them in discussion of the neural net prior. I think it’s important to mention that here.
I signed an amicus brief supporting Anthropic’s right to do business without governmental retaliation. As an AI expert, I attest that Anthropic’s technical concerns are legitimate, and no laws were designed to protect against AI analysis of surveillance data.
Even though I work at a competing lab (Google DeepMind), I’m proud of Anthropic for taking a stand against unlawful retaliation and immoral demands.
(I speak only for myself, not my employer.)
Realized I made a prediction on Jan. 1 about whether I’d leave GDM in 2026:
My Fatebook question (shared with permission from Peter).