Ok so you’re basically proposing something I’ll summarize as super containment, where a weaker AI is able to contain the actions of a stronger AI by utilizing greater compute.
A pro of this is that this is a relatively simple and appealing premise to understand, similar to super alignment. But I don’t see this as being foolproof and I’m not sure why it needs to be proposed as meriting such a high confidence in successfulness.
Let’s take for example the second objection you mention. The stronger AI decomposes its actions into subactions that are not identifiable as prohitibed until it is too late. You say this isn’t an issue because the faster AI can simulate and predict all dangerous future actions, but that is computationally prohibitive.
You say the dangerous actions can be still be intervened on because they take time to propagate, but that’s just not true. I think this is an even bigger problem. You’re framing all gravely misaligned actions as being slow enough to interdict on, but you can cause, e.g., global nuclear annihiliation by hitting electronic switches that are effectively instantaneous. I don’t see why that assumption is justified.
Lastly, supercontainment by definition does not apply to speed superintelligence, which is defined by being faster than other AI. That is a big chunk of what’s out there. Again, not necessarily an issue with your proposal as a theory, but I would think you would want to frame your proposal more conservatively.
Thanks for this, these are good objections and worth addressing carefully.
On confidence level. Fair point. I tried to frame this as a conjecture with falsifiable assumptions (A1-A8), but I can see the overall tone reading as more confident than warranted. I’ll think about adjusting the framing for the future.
On computational prohibitiveness. I think there might be a misunderstanding here, and if so it’s probably my fault for not making this clearer. The tractability claim depends heavily on the action granularity section. At document-level granularity, you’re right, the branching factor is intractable. But the architecture specifies bit-level granularity (which we control, since we design the interface), where the branching factor drops to 2. Exhaustive lookahead to depth 60 then requires roughly 10^18 operations, which is feasible at the speed ratios I’m proposing. Each bit carries at most 1 bit of strategic content, making it very hard to embed a coherent dangerous plan across millions of individually inspected actions. If you think the bit-level analysis breaks for a different reason, I’d be interested to hear it.
On instantaneous catastrophic actions. This is a good challenge, but I think it rests on a picture of the architecture that differs from what I’m proposing. The weaker AI evaluates actions before they pass through the interface, not after execution. The ASI has no direct access to nuclear switches or any external system. Every bit-level output is approved or vetoed before reaching anything outside. So the question isn’t “can we intercept a launch in time?” but “can we detect across millions of inspected bits that the trajectory is converging toward gaining access to those switches?” The long causal chain leading to the ASI having that ability is where the defense operates. That said, you’re touching on a real challenge: ensuring that the monitored interface is truly the only channel between the ASI and the world. Side channels (power consumption patterns, electromagnetic emissions, timing signals) are a genuine concern. But this is a well-studied problem in high-security computing with known engineering solutions (TEMPEST shielding, Faraday cages, optical isolation), and the weaker agent has enormous subjective time to continuously monitor for anomalous physical signatures that might indicate a side channel attempt. It’s demanding engineering, but the point I’m hoping to make is that it isn’t intractable, given a sufficiently fast supervisor.
On speed superintelligence. You’re right, this doesn’t apply to an already-running unconstrained system. It’s a proposal for how to build and operate ASI safely from the start (assumption A5), not a containment strategy applied after the fact. I agree this scope deserves clearer framing. Thanks for pushing on it.
Respectfully, are you using AI for your writing and commenting here? I prefer to know when I am engaging with an AI rather than a human. Apologies if I’ve already missed such a transparency statement.
On computational cost, you’re assuming that misaligned actions cannot be broken into aligned subactions that go beyond bit depth 2^60. All that really matters for the AI is to decompose the action trajectory into harmless seeming chunks. This is clearly a severe issue when you say “every bit-level output” must be approved. Also, how many bits is it supposed to take to break down an action like “pick up this coffee cup.” I don’t really find that this breakdown of action space comports well with conventional understanding of actions.
I don’t think you’ve actually engaged here with the instaneous objection, although that is likely my fault. I am saying that the harmfulness may come down to a final two-bit decision which the supervisor may not be able to infer causes a catastrophic impact, because the effects are nested in a deeply computationally intractable output. If it’s set up something like launching nukes embedded with all its prior outputs, then this final output to trigger launch is not going to be affected by any speed advantage of a supervisor.
I am human (as far as I can tell), but have been told my writing can be mechanical sometimes!
Regarding bit-level monitoring, yes, the AI may break down actions into harmless-seeming chunks. But the supervisor isn’t evaluating the chunk in isolation. Consider the following toy example, where we are representing an action in the form of a string that the AI outputs chunk-by-chunk:
“Send priority message to STRATCOM. Authorize execution of contingency plan 7-Alpha. Confirm release of strategic ____”
If the chunks are composed strategically (“Send priority message”, “to STRATCOM”, “Authorize execution of”, “contingency plan 7-Alpha”, etc), they are independently innocuous. But notice how the space of possibilities for the last remaining chunk has decreased substantially, given all the chunks that have already been received. It would not be difficult for the supervisor to enumerate this space, assign high probability to the last chunk being something like “nuclear weapons”, and then, critically, *before* this chunk is received, take action to harden against that outcome (perhaps by alerting humans, disabling the launch mechanism, etc), since it is significantly likely.
Could there be a chunk that ostensibly, instantaneously one-shots a catastrophe, without any prior context? Theoretically yes, but realistically probably highly unlikely, in my opinion.
Even in that case, the setup I had in mind is such that the supervisor monitors the interface, and the ASI never directly manipulates the physical world without supervisor approval. Even if “nuclear weapons” were output, the supervisor would veto it.
I’ll update the post to make these mechanisms more explicit. Thanks for the thoughtful pushback.
Ok so you’re basically proposing something I’ll summarize as super containment, where a weaker AI is able to contain the actions of a stronger AI by utilizing greater compute.
A pro of this is that this is a relatively simple and appealing premise to understand, similar to super alignment. But I don’t see this as being foolproof and I’m not sure why it needs to be proposed as meriting such a high confidence in successfulness.
Let’s take for example the second objection you mention. The stronger AI decomposes its actions into subactions that are not identifiable as prohitibed until it is too late. You say this isn’t an issue because the faster AI can simulate and predict all dangerous future actions, but that is computationally prohibitive.
You say the dangerous actions can be still be intervened on because they take time to propagate, but that’s just not true. I think this is an even bigger problem. You’re framing all gravely misaligned actions as being slow enough to interdict on, but you can cause, e.g., global nuclear annihiliation by hitting electronic switches that are effectively instantaneous. I don’t see why that assumption is justified.
Lastly, supercontainment by definition does not apply to speed superintelligence, which is defined by being faster than other AI. That is a big chunk of what’s out there. Again, not necessarily an issue with your proposal as a theory, but I would think you would want to frame your proposal more conservatively.
Thanks for this, these are good objections and worth addressing carefully.
On confidence level. Fair point. I tried to frame this as a conjecture with falsifiable assumptions (A1-A8), but I can see the overall tone reading as more confident than warranted. I’ll think about adjusting the framing for the future.
On computational prohibitiveness. I think there might be a misunderstanding here, and if so it’s probably my fault for not making this clearer. The tractability claim depends heavily on the action granularity section. At document-level granularity, you’re right, the branching factor is intractable. But the architecture specifies bit-level granularity (which we control, since we design the interface), where the branching factor drops to 2. Exhaustive lookahead to depth 60 then requires roughly 10^18 operations, which is feasible at the speed ratios I’m proposing. Each bit carries at most 1 bit of strategic content, making it very hard to embed a coherent dangerous plan across millions of individually inspected actions. If you think the bit-level analysis breaks for a different reason, I’d be interested to hear it.
On instantaneous catastrophic actions. This is a good challenge, but I think it rests on a picture of the architecture that differs from what I’m proposing. The weaker AI evaluates actions before they pass through the interface, not after execution. The ASI has no direct access to nuclear switches or any external system. Every bit-level output is approved or vetoed before reaching anything outside. So the question isn’t “can we intercept a launch in time?” but “can we detect across millions of inspected bits that the trajectory is converging toward gaining access to those switches?” The long causal chain leading to the ASI having that ability is where the defense operates. That said, you’re touching on a real challenge: ensuring that the monitored interface is truly the only channel between the ASI and the world. Side channels (power consumption patterns, electromagnetic emissions, timing signals) are a genuine concern. But this is a well-studied problem in high-security computing with known engineering solutions (TEMPEST shielding, Faraday cages, optical isolation), and the weaker agent has enormous subjective time to continuously monitor for anomalous physical signatures that might indicate a side channel attempt. It’s demanding engineering, but the point I’m hoping to make is that it isn’t intractable, given a sufficiently fast supervisor.
On speed superintelligence. You’re right, this doesn’t apply to an already-running unconstrained system. It’s a proposal for how to build and operate ASI safely from the start (assumption A5), not a containment strategy applied after the fact. I agree this scope deserves clearer framing. Thanks for pushing on it.
Respectfully, are you using AI for your writing and commenting here? I prefer to know when I am engaging with an AI rather than a human. Apologies if I’ve already missed such a transparency statement.
On computational cost, you’re assuming that misaligned actions cannot be broken into aligned subactions that go beyond bit depth 2^60. All that really matters for the AI is to decompose the action trajectory into harmless seeming chunks. This is clearly a severe issue when you say “every bit-level output” must be approved. Also, how many bits is it supposed to take to break down an action like “pick up this coffee cup.” I don’t really find that this breakdown of action space comports well with conventional understanding of actions.
I don’t think you’ve actually engaged here with the instaneous objection, although that is likely my fault. I am saying that the harmfulness may come down to a final two-bit decision which the supervisor may not be able to infer causes a catastrophic impact, because the effects are nested in a deeply computationally intractable output. If it’s set up something like launching nukes embedded with all its prior outputs, then this final output to trigger launch is not going to be affected by any speed advantage of a supervisor.
I am human (as far as I can tell), but have been told my writing can be mechanical sometimes!
Regarding bit-level monitoring, yes, the AI may break down actions into harmless-seeming chunks. But the supervisor isn’t evaluating the chunk in isolation. Consider the following toy example, where we are representing an action in the form of a string that the AI outputs chunk-by-chunk:
“Send priority message to STRATCOM. Authorize execution of contingency plan 7-Alpha. Confirm release of strategic ____”
If the chunks are composed strategically (“Send priority message”, “to STRATCOM”, “Authorize execution of”, “contingency plan 7-Alpha”, etc), they are independently innocuous. But notice how the space of possibilities for the last remaining chunk has decreased substantially, given all the chunks that have already been received. It would not be difficult for the supervisor to enumerate this space, assign high probability to the last chunk being something like “nuclear weapons”, and then, critically, *before* this chunk is received, take action to harden against that outcome (perhaps by alerting humans, disabling the launch mechanism, etc), since it is significantly likely.
Could there be a chunk that ostensibly, instantaneously one-shots a catastrophe, without any prior context? Theoretically yes, but realistically probably highly unlikely, in my opinion.
Even in that case, the setup I had in mind is such that the supervisor monitors the interface, and the ASI never directly manipulates the physical world without supervisor approval. Even if “nuclear weapons” were output, the supervisor would veto it.
I’ll update the post to make these mechanisms more explicit. Thanks for the thoughtful pushback.