The way I’m thinking about it is, various inputs are scaling exponentially (e.g. compute, data, human labor) to produce the trend we see. Obviously, when input scaling slows down, progress will slow too, creating a sort of sigmoid.
But I’m interested in what happens if the inputs continue to scale at approximately the same rate. I’ll then adjust up or down from there depending on a separate projection of how the inputs will scale or slow.
Do you expect a sigmoid even if the inputs continue to scale at approximately the same rate? If so, why? Do you think there is some inherent limit on the agentic coding task horizon length of AIs trained within roughly the current paradigm?
The current paradigm seems likely to become a sigmoid before the 8 hour mark. I tried to place a bet on this with the AI 2027 authors but no one accepted. (At this moment I’m cash-poor so it would have to be a small amount or just provide me liquidity on the manifold market I linked)
This is mostly because I don’t expect “neuralese” to just work, as indeed it has not yet (?) despite one paper (which was cited in AI 2027) that apparently hasn’t replicated, because tokens are too low bandwidth. I also think reinforcement learning remains hard even if you have a good predictive model of text on the internet.
I expect these problems to kick in around the 4-16 hour mark, when it’s necessary to build a sophisticated mental model of a particular problem.
We’re already seeing that LLMs may not speed up developer productivity and do not produce usable PRs on large projects. These limitations will perhaps become blocking at time horizons slightly longer than the current SOTA, when you can’t just one-shot a project.
On the other hand, the slower exponential trend has proven more robust than I expected. I think that the faster exponential for reasoning models is dying off (with Grok 4 and GPT-5) and the slower exponential dies off next, but I am becoming slightly less confident of this on balance.
Oh cool, yeah I’m happy to bet with you on that I think. What would the exact terms be? 8+ hour horizon length AI by mid-2027?
I sure hope neuralese doesn’t work, and agree it hasn’t been working so far. In general I really hope you are right about the incoming wall that deep learning is about to hit, but I don’t think you are, deep learning has smashed through so many alleged walls recently.
That’s way below the slower exponential, I think I need slightly more favorable terms for 50:50.
I would take that bet if EITHER the resolution is end of March 2027 rather than mid-2027 or the number is 12+, both of which are still substantially below the slower exponential.
Since all these dates are pretty far out I’d bet, say, 250 USD.
I also hope I’m right, but I don’t necessarily except deep learning to hit a wall, only the current paradigm within deep learning.
Is it way below? Eyeballing the original graph from the paper, it seems like it would just be slightly below the slower exponential:
Anyhow I’m happy to take the bet at 50:50 odds for 8 hours at end of March 2027. I’m not confident I’ll win, but I think I’m somewhat more likely than not to win. 250 USD sounds good to me.
Right, I am projecting the slower exponential growth rate forward starting from this point.
I am also not confident that I will win, but I accept, let’s make a prediction market to track this specific bet (I am curious what the manifold odds will look like).
The 4-month doubling trend implies getting 8h+ horizon length by early 2026 and an order of magnitude more by mid-2027. If the best time horizon length in mid-2027 would be 9h, would you feel like you had won the argument, even if you had won the bet?
I interpret my opponent here as saying things will go slower than the 7-month doubling trend, not as saying that things will go slower than the 4-month doubling trend.
That said if it’s 9hr then it’s basically right on the line between winning and losing the bet, only a very weak winning of the bet, so no I wouldn’t really think of myself as having won the argument.
Exponential and superexponential are not the only plausible options here.
I’m still expecting a sigmoid.
Exponential is starting to look more convincing though.
The way I’m thinking about it is, various inputs are scaling exponentially (e.g. compute, data, human labor) to produce the trend we see. Obviously, when input scaling slows down, progress will slow too, creating a sort of sigmoid.
But I’m interested in what happens if the inputs continue to scale at approximately the same rate. I’ll then adjust up or down from there depending on a separate projection of how the inputs will scale or slow.
Do you expect a sigmoid even if the inputs continue to scale at approximately the same rate? If so, why? Do you think there is some inherent limit on the agentic coding task horizon length of AIs trained within roughly the current paradigm?
The current paradigm seems likely to become a sigmoid before the 8 hour mark. I tried to place a bet on this with the AI 2027 authors but no one accepted. (At this moment I’m cash-poor so it would have to be a small amount or just provide me liquidity on the manifold market I linked)
A priori I expect the current model to fail at continual learning: https://www.lesswrong.com/posts/vvgND6aLjuDR6QzDF/my-model-of-what-is-going-on-with-llms
This is mostly because I don’t expect “neuralese” to just work, as indeed it has not yet (?) despite one paper (which was cited in AI 2027) that apparently hasn’t replicated, because tokens are too low bandwidth. I also think reinforcement learning remains hard even if you have a good predictive model of text on the internet.
I expect these problems to kick in around the 4-16 hour mark, when it’s necessary to build a sophisticated mental model of a particular problem.
We’re already seeing that LLMs may not speed up developer productivity and do not produce usable PRs on large projects. These limitations will perhaps become blocking at time horizons slightly longer than the current SOTA, when you can’t just one-shot a project.
On the other hand, the slower exponential trend has proven more robust than I expected. I think that the faster exponential for reasoning models is dying off (with Grok 4 and GPT-5) and the slower exponential dies off next, but I am becoming slightly less confident of this on balance.
Oh cool, yeah I’m happy to bet with you on that I think. What would the exact terms be? 8+ hour horizon length AI by mid-2027?
I sure hope neuralese doesn’t work, and agree it hasn’t been working so far. In general I really hope you are right about the incoming wall that deep learning is about to hit, but I don’t think you are, deep learning has smashed through so many alleged walls recently.
That’s way below the slower exponential, I think I need slightly more favorable terms for 50:50.
I would take that bet if EITHER the resolution is end of March 2027 rather than mid-2027 or the number is 12+, both of which are still substantially below the slower exponential.
Since all these dates are pretty far out I’d bet, say, 250 USD.
I also hope I’m right, but I don’t necessarily except deep learning to hit a wall, only the current paradigm within deep learning.
Is it way below? Eyeballing the original graph from the paper, it seems like it would just be slightly below the slower exponential:
Anyhow I’m happy to take the bet at 50:50 odds for 8 hours at end of March 2027. I’m not confident I’ll win, but I think I’m somewhat more likely than not to win. 250 USD sounds good to me.
Right, I am projecting the slower exponential growth rate forward starting from this point.
I am also not confident that I will win, but I accept, let’s make a prediction market to track this specific bet (I am curious what the manifold odds will look like).
Deal. Thanks!
Please let me know soon if you want any part of the description modified.
The 4-month doubling trend implies getting 8h+ horizon length by early 2026 and an order of magnitude more by mid-2027. If the best time horizon length in mid-2027 would be 9h, would you feel like you had won the argument, even if you had won the bet?
I interpret my opponent here as saying things will go slower than the 7-month doubling trend, not as saying that things will go slower than the 4-month doubling trend.
That said if it’s 9hr then it’s basically right on the line between winning and losing the bet, only a very weak winning of the bet, so no I wouldn’t really think of myself as having won the argument.
That’s also my impression: https://www.lesswrong.com/posts/KrgBkqeChtAWuPsLP/what-llms-lack