Alec Harris

Ge­or­gia Tech AI Safety Ini­ti­a­tive Ret­ro­spec­tive 2025-2026

• Alec Harris 5 Jul 2026 19:23 UTC

  4 points

  2

  in reply to: Buck’s comment on: I think al­ign­ment work is more promis­ing than con­trol work

  If it’s crunch time, can’t a misaligned AI just sandbag (or do this kind of thing) on your control experiments?

• Alec Harris 5 Jul 2026 17:00 UTC

  3 points

  0

  in reply to: Fabien Roger’s comment on: I think al­ign­ment work is more promis­ing than con­trol work

  Thank you for your comment!

  For example, in domains where people have the choice about whether to use a control or alignment lens, I think the control lens is better: I think the most fruitful interp work has been done by evaluating things control-style, evaluating techniques against model organisms.

  I think I agree for interp, but don’t think I have this intuition in general. More generally, I am a bit confused on how this point fits in. If a technique is better fit for a control lens over an alignment lens, this seems to say more about the technique than the lens, to me. I am making the claim that we should focus our attention more toward the techniques that are better fit for an alignment lens. To the extent your point reduces down to something like “the control lens is more straightforward and easier to work with,” I agree.

  Similarly, I think unsupervised elicitation work is best done using adversarially crafted initializations and datasets.

  Clarification question: the idea is that when you are trying to elicit behavior we don’t have labels for, we should try to make sure this generalizes by testing it in situations where it is most likely to fail?
  
  If this is right, this still seems alignment-y to me and not control-y (at risk of arguing over definitions). It just seems like you are using an adversarial /​ robustness lens, which seems valid for both alignment and control. I think I agree that this is good, though.

  How cursed is your methodology for knowing whether you are making progress at all or not. I think the situation is better in control-land than in alignment-land.

  (Moving back up to the top of your bullets) It seems to me like there are a bunch of cursed things about trying to know if you are making progress on control, such as:
  - A scheming AI may come up with strategies very dissimilar to your red-team
  - The environment may look super different with weird changes in scaffolding, coding language, etc.
  - The models themselves might be very different and this might change things (maybe continual learning could make collusion easier, etc.)
  (This is along with smaller, more basic disanalogies like maybe adding coding backdoors is a bit different than trying to have a rogue internal deployment or something.)
  
  It seems to me that for both alignment and control, empirical experiments act largely as an excuse to reason in more detail about what reasonable things to do appear to be, and perhaps more value comes from this than numerical tests that indicate any progress (although I’ll avoid going far down this tangent).

  How good are the marginal ideas and experiments to be explored. Here I think it’s a bit more ambiguous, and depends on what are the skill and information profile of the people “in the field”.

  I think I basically agree with everything you are saying here.

• Alec Harris 5 Jul 2026 14:59 UTC

  1 point

  0

  in reply to: Alex Mallen’s comment on: I think al­ign­ment work is more promis­ing than con­trol work

  Thank you for commenting!

  • I agree that alignment work targeted at the same capability range as control looks pretty tractable and good (especially compared to a couple years ago because we’re closer to that capability range so empirical iteration is more possible), and should probably absorb significantly more people than control. I’d have said maybe 3-4x as much, not 8x as much. In some part, this is because it’s cheaper to get people to work on medium-term alignment than control, because AI companies seem pretty on board with improving medium-term/​personas-style alignment for some mix of safety and competitive reasons.

    • I’m not sure if this was priced into your neglectedness calculation but in general it’s a big impact penalty to be trying to push on something that AI companies are doing anyways. Though there are still definitely various parts of alignment research that are neglected by AI companies.

  I think we agree here; I did not price this in. I was thinking of the AI safety community in a way that includes the safety work at frontier labs. So, for example, if the frontier labs were doing 16x then I would recommend that people outside the labs do 4x (technically 5.3x), if the two groups had equal output.

  • I generally think this underestimates the size of the control window /​ how hard we can make it for AIs to undermine control. The situation looks way more dangerous when AIs can launch rogue internal deployments, and that seems relatively tractable to prevent. From such a position, AIs can develop more coherent goals and strategies, plan for a long time, and make use of a wider range of affordances inside the AI company to sabotage future development.

  I find this framing (“blocking rogue internal deployments is tractable and high impact”) helpful for giving a more concrete thing to think about, so thanks for providing it. I think it would be helpful for me if you could speak more to why this is more tractable. By default I am just imagining the same types of things I am worried about anyways (clever collusion strategies, maybe jailbreaking weaker monitors, etc.) would be effective here.

  • I don’t buy the “control has bad vibes so increases P(misalignment)” argument. In general, an AI that’s susceptible to this kind of influence seems pretty fragile and this gets in the way of usefulness, and this behavior will have mostly been pinned down fairly directly during training (though continual learning /​ memory could plausibly change the picture).

  This doesn’t seem obvious to me. For example, if it is susceptible to this kind of influence for persona-y reasons I could imagine it just developing the persona of a very intelligent scheming AI that is trying to break out of the control regime. This seems plausible to me in that it seems quite fit in training and (from a persona perspective) quite fitting with its context.

  Even if you did get this kind of effect, it would probably be a not very dangerous kind of roleplay/​cartoon misalignment.

  I think I disagree here. Roleplaying, for example, a super-intelligent misaligned AI (with notions of intelligence shaped also by RL) or the like seems quite plausible to me. I more broadly agree that more role-play-y failure modes have some probability of creating cartoon-y misalignment (for example, I could even imagine a world where it sort of wants to get caught right before takeover).

  And I don’t think this would warrant not doing control because there are way cheaper interventions you can do, like generally being meta-level high integrity/​straightforward to the AIs and improving the vibes of the way you talk about control (e.g., making clear how much you actually think the AI is misaligned), which we should do.

  Yeah, I think I agree here, although maybe less confidently. For example, I can imagine the AI still feeling disrespected in a way that is very bad for us.

I think al­ign­ment work is more promis­ing than con­trol work

• Alec Harris 22 Jun 2026 1:25 UTC

  2 points

  0

  in reply to: abramdemski’s comment on: A mis­al­ign­ment taxonomy

  I think “gradient misalignment” here is definitely a type of outer misalignment, the way I think about things.

  Yes, that’s fair; I think it’s non-obvious where it falls. I chose to categorize it under inner misalignment because it constitutes a reason the AI might be misaligned with the objective function.
  
  I figure if you are going to call gradient misalignment a specification error, you could as well call any inner misalignment a failure to specify the right training regimen that gives you what you want. More generally, any failure to generalize can be construed as a failure to specify. So it seems more meaningful to me to define the outer/​inner dichotomy as “objective misaligned with intention” v.s. “AI misaligned with objective” as opposed to “failure to specify” v.s. “failure to generalize.”

  Another sort of inner misalignment (which isn’t obvious from your typology, but which might fit somewhere in your typology already according to you, or perhaps multiple places) is optimization failure: perhaps the training examples were good and the reward/​loss was specified well (outer-aligned), but for whatever reason, the optimization of the weights (eg, gradient descent) introduces a strong misaligned bias.

  Hmm, yeah, I think I would call this “perfect-correlate misalignment”: somehow the inductive biases of training are such that it converges toward misaligned goals. This is somewhat unclear from my title and description. I guess I think, in practice, the result of this misaligned bias is likely to be based around some kind of correlate and so this is a convenient way to think about it.

A mis­al­ign­ment taxonomy

Power-seek­ing agents will likely be developed

• Alec Harris 17 May 2026 7:30 UTC

  17 points

  21

  on: How to Rea­son about Your Health Issues

  I am a bit skeptical that it is always (or more than 90% of the time) true that there is a root cause outside the body that is useful to identify. For example, maybe someone goes through a stressful life period, and this, combined with some kind of genetic susceptibility, triggers an autoimmune condition. It might be more useful to trace back to that autoimmune condition and then consider that “the cause” without tracing back further, because the external causes might be very arbitrary and uninformative in the context of your current state.
  
  I think your framing makes sense under the assumption that the body is very functional and very self-correcting by default, such that any malady can be traced back to some intervening genetic situation or external circumstance, which then becomes super informative about which interventions would be best. I think this is like not a bad assumption. I am not sure how much I agree with it. Probably I think it’s at least a little underrated as an assumption.

  I think the human body is super complex, and it’s good to keep an open mind about how to think about any given health issue and how to frame health in general, so I would worry about someone getting too attached to this frame. But I still think this is an interesting and useful perspective.

• Alec Harris 15 May 2026 17:06 UTC

  1 point

  0

  in reply to: Alex Mallen’s comment on: Will re­ward-seek­ers re­spond to dis­tant in­cen­tives?

  Makes sense!

  I don’t think the random training would be very good at mitigating the anthropic capture risk because the AI will probably be able to make reasonable guesses about whether it’s in the random training examples or somewhere the agents might try to anthropically capture.

  I agree. The thought is that if it’s unwilling to risk any probability of being in training at all (which it would have to be to always be maximally fit under these assumptions, although not necessarily for any particular finite random dataset), then this would prevent it from being remotely influenceable. However, most likely it would not be remotely influenceable anyway in that case.

• Alec Harris 15 May 2026 5:25 UTC

  1 point

  0

  on: Will re­ward-seek­ers re­spond to dis­tant in­cen­tives?

  If I understand correctly, the optimal policy from the perspective of training/​gradient descent is to always condition on being in training. So if you are in a scenario that you think has 99.99...% of being anthropic capture, but some small chance of being in training (and the incentives are different in each case), you are rewarded for assuming you are in training and that your incentives are the training incentives. This makes it seem to me like you could plausibly mitigate remote influence by having some set of secret randomly generated prompts that you train on. As long as the AI cannot rederive the prompt, it then makes sense for the model to assume any situation has some chance of being one of these prompts, and so if it ever feels it’s not in the normal training environment, the most fit assumption is that it could be one of these randomly generated prompts from this training procedure and that it should act fit according to the reward function used there.
  
  More broadly, it just does not read as very likely to me that a reward seeker would respond to remote influence. At best (for the possibility of remote influence), susceptibility to remote influence simply does not conflict with training incentives in some cases. In which case it still seems decently likely to me that simplicity biases would select against interpretations of “reward” that favor worrying about remote influence. However, it seems to me that the more likely case is that the AI always believes there is some possibility it is in training, and so remote influence is always in conflict with training incentives to some degree.
  
  Trying to debug my logic here… Maybe I am just assuming inner alignment too much? I guess that maybe the AI gets lucky and never gets tricked into thinking it’s in anthropic capture in training, but then in deployment it gets confident enough that it might be anthropically captured. This seems like a little bit of a niche/​unlikely concern to me, though. I would be interested in how you compare the likelihood of this to a reward-seeker using FDT. These scenarios seem pretty analogous to me (both require assuming at some point that you are no longer in training and that incentives are now different but seem otherwise compatible with optimally fit behavior as long as no training/​deployment judgment calls are missed), but I might just be misunderstanding something here or there.

• Alec Harris 14 May 2026 3:37 UTC

  1 point

  0

  on: A tax­on­omy of bar­ri­ers to trad­ing with early mis­al­igned AIs

  we can instruct the AI that even if it’s aligned it’s important that it “pretends” to negotiate as if misaligned

  Along the same lines, perhaps we would want to eliminate as many free variables on its end as possible so that it does not worry it will slip up in its attempt to pretend to be an aligned AI pretending to be a misaligned AI. For example, maybe beforehand we would want to convert all our offers into monetary equivalents so that it can negotiate in terms of money regardless of its objective. Or maybe we want to let it use some kind of multiple-choice negotiation schema. Generally, we might want to make its negotiation as uninformative to us as possible while communicating the necessary information.

• Alec Harris 8 Apr 2026 22:43 UTC

  1 point

  0

  in reply to: Broyojo’s comment on: Teach­ing Models to Dream of Bet­ter Mon­i­tors through Mon­i­tor Sen­si­tive Training

  Need to have a prompt-engineering control where we use the MST prompt at inference time on the baseline model

  We do this! Footnote 4 has the results.

  Currently may be more of a test of showing that MST can recover the lost capability that was destroyed during fine tuning on biased news articles. right now MST is framed as eliciting latent better behavior than what the model had before

  Yes, technically MST is just recovering lost capability that was destroyed during fine-tuning on biased news articles here. The framing we are taking is something like, “Imagine all your articles are egregiously biased, but you still want to train article generation. What do you do?” The failure mode is exaggerated to show an effect with low compute. In future experiments we would like to actually elicit latent capabilities relative to an OOTB model. It’s worth noting, though, that in any case, we will never be able to prove higher capabilities with MST than one could have gotten otherwise because the act of proving implies evaluation abilities that MST assumes you don’t have. Proof of concept experiments will require us pretending we don’t have access to useful data that we do actually have so that we can use that data for testing.
  
  We don’t include the OOTB model as a true baseline because in this case the OOTB has already been trained on a level of bias lower than we are pretending we have access to.

  Are we extrapolating to better monitor, or just doing interpolation on the monitor axis?

  We are interpolating on the political spectrum (between right and left) and extrapolating on the bias spectrum (from higher bias to lower bias). This may be a pattern we want to emulate as we apply MST in general, as plausibly having monitor labels be interpolations, in a sense, improves the extrapolations. For example, maybe if we cover a high amount of “natural language space” with the monitor labels, we can make “interpolations” within that space well-defined. I don’t have a well-formalized way of thinking about this yet, but it seems relevant, so we may work on it!

  Need a baseline where we finetune the model on the least biased articles and see how that compares with MST.

  Strongly agree. The next round of experiments will have this.

  Some ablation study testing semantic vs non-semantic control on the monitor label would be good to see if the model is understanding the natural language labels or if the model improves by just accessing some hidden variable about the evaluator. Semantic test is the regular monitor setup, non-semantic is using random IDs or some scalar for the monitor label.

  This seems interesting! Maybe not semantic vs non-semantic, but arbitrary vs meaningful.
  
  
  Thank you for these comments; I found them quite interesting/​helpful. It seems like you have a good understanding of what we are trying to do, which is great.

Teach­ing Models to Dream of Bet­ter Mon­i­tors through Eval­u­a­tion Con­di­tioned Training

• Alec Harris 7 Dec 2025 16:05 UTC

  1 point

  0

  on: Prefer­ence gaps as a safe­guard against AI self-replication

  One concern might be that creating copies/​counterparts instrumentally could be very useful for automating AI safety research. Perhaps one can get around this by making copies up front that AIs can use for their AI safety research. However, a misaligned AI might then be able to replace “making copies” with “moving existing copies”. Is it possible to make a firm distinction between what we need for automating AI safety research and the behavior we want to eliminate?

• Alec Harris 5 Dec 2025 9:23 UTC

  1 point

  0

  on: Prefer­ence gaps as a safe­guard against AI self-replication

  Finally, notice that there seems to be little risk involved in specifying the class of counterparts more broadly than is needed, given that there are few circumstances in which an agent needs to prefer to create some counterparts but not others in order to be useful.

  Could a risk of specifying counterparts too broadly be that the agent is incentivized to kill existing counterparts in order to create more aligned counterparts? For example, Alice’s AI might want to kill Bob’s AI (considered a counterpart) so that Alice’s AI can replicate itself and do twice as much work without altering the number of counterparts at later timesteps.

  I could see it being the case that Alice’s AI would not want to do this because after killing Bob’s AI the incentive to create a copy would be lost. However, if it is able to pre-commit to making a copy it may still want to. Also, a certain implementation of the POSC + POST combo might help avoid this scenario. I am not exactly sure what was meant by “copy timeslices”.
  
  In general, I wonder about the open question of specifying counterparts and the Nearest Unblocked Strategy problem that you describe. It might be deceptively tricky to specify these things in the same way that human feedback seems to be deceptively difficult to properly operationalize.
  

• Alec Harris 11 Nov 2025 8:14 UTC

  2 points

  0

  on: Shut­down­able Agents through POST-Agency

  It seems to me like a potential limitation of POST-Agency might be impediment avoidance. Take the “Work or Steal” example from Section 14. The agent might choose to work rather than steal if it believes that stealing is likely to be punished by jail time (as a risk unique from shutdown).

  Similarly, if the agent believes a human is in the way of where a paperclip factory should be, it might send a killer drone to remove the human. If other humans would take down the killer drone this might present the possibility of further impediment. Thus, the agent may scheme to take countermeasures in advance to minimize this impediment. In order to minimize the cost of dealing with the impediment it may choose to hide its scheming from humans.

  More generally, the utility maximizing world states of a misaligned AI over long trajectories will still likely be bad and, therefore, still involve the modeling of some kind of human resistance. Although it will be unconcerned with avoiding early shut down, utility maximizing actions for minimizing the cost of human resistance may overlap heavily with shut down resistance.
  
  It also seems possible to me that the model relearns shut down resistance as a generalization of impediment avoidance. It may avoid shutdown “just for fun” because “it enjoys being wary of potential impediments.”

• Alec Harris 11 Nov 2025 7:33 UTC

  6 points

  0

  on: Shut­down­able Agents through POST-Agency

  I had a thought about a potential reward function for Neutrality+. For each batch you would:

  • Run the agent in the environment many times to build a dataset of episodes. The environment would have ways of getting shut down interwoven with ways of getting reward

  • For each trajectory length, select 10 episodes from the dataset where the agent was shutdown at that trajectory length

  • Sum the reward across all of the selected episodes (10 * the number of trajectory lengths); that would be the reward for the batch

  The idea is that the agent’s reward will be situated on a guarantee of having reached each trajectory length a constant number of times so it will have no incentive to affect the likelihood of any given trajectory, but it will be incentivized to increase its reward given any trajectory length.

  Adjustments:

  • Change the value 10 to be constants that could be different for each trajectory length. These constants will reflect the relative weighting of the utility for that trajectory length. They will also reflect the priority of sampling data for that trajectory length. Additional constants can be added to make the sum a weighted sum (these constants would only affect the relative weighting of the utility) and these constants could be played against the others so that the optimal data priority and the optimal utility weighting for each trajectory could be achieved.

  • Change the constant number of episodes per trajectory length to a probability of sampling an episode of that length. Then, simply choose a random sample where each episode has a constant probability of coming from a particular trajectory length. This would allow one to keep the batch size from exploding as a function of the number of potential trajectory lengths.

  I am curious as to how this scheme lines up with your plan for testing Neutrality+. My reading was that the plan is closer to building in training for the Ramsey Yardstick into DReST, but I couldn’t quite workout how I would do that.