OpenAI claims to have paused frontier RL training for now. Altman stated on X:
We have paused some frontier RL training to ensure that we can meet the appropriate alignment, security and monitoring standards for the new level of capabilities in front of us. Model progress is now extremely rapid, and we always said we would take action if we felt that model capabilities were outstripping the pace of safety and alignment.
We care very deeply about AI safety. We believe the entire field will have to coordinate on shared safety standards, but will act unilaterally in the meantime.
We expect confidence in safety to increasingly set the pace of AI progress. We are optimistic about the alignment work we are doing, and we remain committed to making frontier capabilities widely available.
Good catch. Also apparently they are only pausing some of their training for two weeks?
As models become more capable, the risks associated with developing and testing them internally also grow.
We temporarily paused reinforcement learning (RL) training on our latest models intended for deployment for two weeks while we hardened and red-teamed our research environments and expanded monitoring coverage.
Our largest planned frontier RL run remains on hold while smaller-scale training and evaluations validate these safeguards and establish more evidence of alignment.
OpenAI isn’t doing as well financially as it would like to to meet investor expectations, so they did something like a pause to provide a covering excuse for this, not because they have any safety-based motivation for doing so.
Prudence is potentially temporary, but incompetence is long-term. Revenues at any given time are usually taken as a projection of future revenues.
They are deliberately pretty vague about when they paused the training, no? Though it’s true that current training doesn’t really affect revenues anyway; only deployed models do. Then again, an even more galaxy-brained take is that they make announces like this in general in order to inoculate against investor expectations, but if they do this often enough it works against point #1.
(I’m not saying that any of these are their actual motivations. I’m just describing how these hypotheses would work.)
We are continuing to invest aggressively in alignment research, increase evaluation coverage, and use what we learn to inform training and safeguards. We plan to share substantially more about our alignment research in the near future, including what we are learning about model behavior and any novel challenges we uncover.
Making AI safer one observed explosion incident after another!
OpenAI claims to have paused frontier RL training for now. Altman stated on X:
pause some frontier RL training
Good catch. Also apparently they are only pausing some of their training for two weeks?
The parallel cynical hypothesis:
https://in.investing.com/news/stock-market-news/openais-q2-revenue-growth-lagged-anthropic-as-losses-deepened-wsj-reports-5562578
whats does this mean? what’s the hypothesis?
OpenAI isn’t doing as well financially as it would like to to meet investor expectations, so they did something like a pause to provide a covering excuse for this, not because they have any safety-based motivation for doing so.
Why do investors care whether the financial underperformance is coming from incompetence or prudence?
The underperformance was in the past whereas the pause/safety actions start now. The latter cannot explain the former.
Prudence is potentially temporary, but incompetence is long-term. Revenues at any given time are usually taken as a projection of future revenues.
They are deliberately pretty vague about when they paused the training, no? Though it’s true that current training doesn’t really affect revenues anyway; only deployed models do. Then again, an even more galaxy-brained take is that they make announces like this in general in order to inoculate against investor expectations, but if they do this often enough it works against point #1.
(I’m not saying that any of these are their actual motivations. I’m just describing how these hypotheses would work.)
Making AI safer one observed
explosionincident after another!