Tentative claim: If looped recurrence increases effective model depth by 8x, CoT monitoring loses (almost?) all of its usefulness
(Epistemic Status: Quick take, I would love to be wrong, this is based on eyeballing graphs and a desire to quickly get an intuition for how bad looped transformers might be)
[EDIT: I think that this was an incorrect framing, and that the better framing is that increasing effective model params by ~100,000x is the scary thing. I also think that looped transformers can only buy a small fractions of that. I remain confused about how much “effective params” increased recurrence buys]
From the no-chain-of-thought time horizons paper: “We fit TH against parameters for 35 open-weight models and find doubling the 50% TH needs a 4.2× increase in total parameters or a 1.3× increase in layers.”
Eyeballing the graph below, it seems like we shoudl expect the most recent models to have a 7-8 doublings gap between CoT and no-CoT performance, corresponding to about (1.3^8)=8.2x increase in layers. That is, if Opus 4 was trained with 6-8x the number of layers (with everything else adjusted for optimality), it would have no-CoT performance equivalent to its actual, current with-CoT performance. This is equivalent to doubling its ability to evade a monitor on SHADE-Arena.
Important Note: It’s not obvious to me at all that a 2x looped transformer is equivalent to doubling the effective model depth. In fact, I’d guess they’re quite different. I’m currently running the no-CoT time horizon suite on Huginn, a public looped transformer, to try to get a better sense for this.
Caveats:
CoT monitoring might still be useful, especially if there is little to no pressure on it, and it doesn’t degrade for other reasons.
Models will still probably need CoT to reach their maximum capabilities
It’s just that lots of tasks (including deceiving monitors) can probably be done at 2026-with-CoT capabilities
I feel like looped recurrence isn’t analogous to the paper’s empirical trendline, which is based on a different scaling process. I’m not sure how to model a process that can be roughly estimated as giving some multiplier in layers while maintaining the number of parameters.
It seems likely to me that it is analogous, in the sense that increasing parameters, or increasing recurrence, or increasing CoT length can all increase the task-horizon length. I think that the main question here is how does recurrence fit into the model? From the test-time compute paper, it seems plausible that
(1) looped recurrence up to a few times can be seen as maybe increasing effective parameter count by a few times, and also
(2) there is a pretty hard ceiling on the amount of extra compute this buys you. There might be very fast diminishing returns on recurrence.
Note that if (2) is true then my above point (using recurrence to 8x effective model depth destroys CoT monitoring) is wrong, or at least the premise is impossible—it might be that an effective 8x depth would be very very bad, but recurrence alone can (as far as we know) only buy labs an effective 2x depth/param multiplier
I find this unintuitive to engage with, I’d be very curious how it applies to specific tasks like multihop reasoning (where the serial nature of the reasoning is very obvious) and coming up with scheming strategies (which is probably what I care about most).
I’d probably rather think about it in terms of, “Tasks that previously required at least N tokens of CoT can now be done without CoT,” rather than this roundabout argument with number of layers needed to double no-CoT time horizons.
I think you’re saying that getting an 8.2x increase in effective depth equals 1.3^8, and each 1.3x in layers yields a doubling in no-CoT time horizon, which covers the 2^8 gap that exists right now between no-CoT and unlimited CoT time horizons.
This yields a conflict between the two interpretations. I’d probably want to interpret 8 doublings in no-CoT serial reasoning ability as enabling hidden reasoning equivalent to using 2^8=256 tokens in current models. Which is probably enough to do lots of scheming reasoning, but not equivalent to unlimited CoT (which is allowed for normal time horizons). I’m not exactly sure how to reconcile these views. Like I said, I find the “Tasks that previously required at least N tokens of CoT can now be done without CoT” framing more intuitive and compelling, so by default I’d take that perspective.
I’d probably want to interpret 8 doublings in no-CoT serial reasoning ability as enabling hidden reasoning equivalent to using 2^8=256 tokens in current models.
This interpretation seems incorrect to me. 8 doublings roughly gives you GPT-5.5′s with-CoT (i.e., unlimited CoT) time horizon, but it certainly can’t do everything it can do with unlimited CoT in 256 tokens. I think the problematic assumption is that one doubling of CoT tokens amounts to one doubling in the time horizon: I wouldn’t expect models to perform twice as well on our single-pass tasks when allowed to output two tokens instead of one.
I think the problematic assumption is that one doubling of CoT tokens amounts to one doubling in the time horizon
I’m not assuming that; I point out the contradiction this leads to in my comment above. The biggest problem with this is that there’s a finite time horizon with unlimited CoT.
My claim is that intuitively, I’d expect that doubling the looping or the number of layers is more similar to doubling the number of CoT tokens than to doubling the time horizon.
Perhaps confusingly, when I said “8 doublings in no-CoT serial reasoning ability,” I meant “whatever vague notion of serial reasoning ability we’re increasing with looping,” not time horizon.
I think looking at the number of effective hidden tokens makes sense, but IMO the time-horizons soubling is a better measure of “how much unmonitorable work can the model do”.
I do, though, think I was wrong to focus on model depth, and that a more correct metric is parameter counts, which would need to increase by a factor of 4.2^8(i.e. 16 doublings) to get unmonitorable models on the scale of today’s frontier models. And since doubling the recurrence depth probably gets you much less than 2x effective params (maybe 1.3-1.7?), the recurrence might actually have to be crazy huge to be worrying by itself (taking these numbers seriously, it would have to recur ~2^32 times!!). Also, recurrence probably has very diminishing returns, and trying to double the recurrence amount more than a few times has a hard usefulness cap.
After thinking about this more, I feel less worried about recurrence?
Tentative claim: If looped recurrence increases effective model depth by 8x, CoT monitoring loses (almost?) all of its usefulness
(Epistemic Status: Quick take, I would love to be wrong, this is based on eyeballing graphs and a desire to quickly get an intuition for how bad looped transformers might be)
[EDIT: I think that this was an incorrect framing, and that the better framing is that increasing effective model params by ~100,000x is the scary thing. I also think that looped transformers can only buy a small fractions of that. I remain confused about how much “effective params” increased recurrence buys]
From the no-chain-of-thought time horizons paper: “We fit TH against parameters for 35 open-weight models and find doubling the 50% TH needs a 4.2× increase in total parameters or a 1.3× increase in layers.”
Eyeballing the graph below, it seems like we shoudl expect the most recent models to have a 7-8 doublings gap between CoT and no-CoT performance, corresponding to about (1.3^8)=8.2x increase in layers. That is, if Opus 4 was trained with 6-8x the number of layers (with everything else adjusted for optimality), it would have no-CoT performance equivalent to its actual, current with-CoT performance. This is equivalent to doubling its ability to evade a monitor on SHADE-Arena.
Important Note: It’s not obvious to me at all that a 2x looped transformer is equivalent to doubling the effective model depth. In fact, I’d guess they’re quite different. I’m currently running the no-CoT time horizon suite on Huginn, a public looped transformer, to try to get a better sense for this.
Caveats:
CoT monitoring might still be useful, especially if there is little to no pressure on it, and it doesn’t degrade for other reasons.
Models will still probably need CoT to reach their maximum capabilities
It’s just that lots of tasks (including deceiving monitors) can probably be done at 2026-with-CoT capabilities
I feel like looped recurrence isn’t analogous to the paper’s empirical trendline, which is based on a different scaling process. I’m not sure how to model a process that can be roughly estimated as giving some multiplier in layers while maintaining the number of parameters.
It seems likely to me that it is analogous, in the sense that increasing parameters, or increasing recurrence, or increasing CoT length can all increase the task-horizon length. I think that the main question here is how does recurrence fit into the model? From the test-time compute paper, it seems plausible that
(1) looped recurrence up to a few times can be seen as maybe increasing effective parameter count by a few times, and also
(2) there is a pretty hard ceiling on the amount of extra compute this buys you. There might be very fast diminishing returns on recurrence.
Note that if (2) is true then my above point (using recurrence to 8x effective model depth destroys CoT monitoring) is wrong, or at least the premise is impossible—it might be that an effective 8x depth would be very very bad, but recurrence alone can (as far as we know) only buy labs an effective 2x depth/param multiplier
I find this unintuitive to engage with, I’d be very curious how it applies to specific tasks like multihop reasoning (where the serial nature of the reasoning is very obvious) and coming up with scheming strategies (which is probably what I care about most).
I’d probably rather think about it in terms of, “Tasks that previously required at least N tokens of CoT can now be done without CoT,” rather than this roundabout argument with number of layers needed to double no-CoT time horizons.
I think you’re saying that getting an 8.2x increase in effective depth equals 1.3^8, and each 1.3x in layers yields a doubling in no-CoT time horizon, which covers the 2^8 gap that exists right now between no-CoT and unlimited CoT time horizons.
This yields a conflict between the two interpretations. I’d probably want to interpret 8 doublings in no-CoT serial reasoning ability as enabling hidden reasoning equivalent to using 2^8=256 tokens in current models. Which is probably enough to do lots of scheming reasoning, but not equivalent to unlimited CoT (which is allowed for normal time horizons). I’m not exactly sure how to reconcile these views. Like I said, I find the “Tasks that previously required at least N tokens of CoT can now be done without CoT” framing more intuitive and compelling, so by default I’d take that perspective.
This interpretation seems incorrect to me. 8 doublings roughly gives you GPT-5.5′s with-CoT (i.e., unlimited CoT) time horizon, but it certainly can’t do everything it can do with unlimited CoT in 256 tokens. I think the problematic assumption is that one doubling of CoT tokens amounts to one doubling in the time horizon: I wouldn’t expect models to perform twice as well on our single-pass tasks when allowed to output two tokens instead of one.
I’m not assuming that; I point out the contradiction this leads to in my comment above. The biggest problem with this is that there’s a finite time horizon with unlimited CoT.
My claim is that intuitively, I’d expect that doubling the looping or the number of layers is more similar to doubling the number of CoT tokens than to doubling the time horizon.
Perhaps confusingly, when I said “8 doublings in no-CoT serial reasoning ability,” I meant “whatever vague notion of serial reasoning ability we’re increasing with looping,” not time horizon.
I think looking at the number of effective hidden tokens makes sense, but IMO the time-horizons soubling is a better measure of “how much unmonitorable work can the model do”.
I do, though, think I was wrong to focus on model depth, and that a more correct metric is parameter counts, which would need to increase by a factor of 4.2^8(i.e. 16 doublings) to get unmonitorable models on the scale of today’s frontier models. And since doubling the recurrence depth probably gets you much less than 2x effective params (maybe 1.3-1.7?), the recurrence might actually have to be crazy huge to be worrying by itself (taking these numbers seriously, it would have to recur ~2^32 times!!). Also, recurrence probably has very diminishing returns, and trying to double the recurrence amount more than a few times has a hard usefulness cap.
After thinking about this more, I feel less worried about recurrence?