Anthropic claims Mythos is able to reliably find exploitable security flaws in lots of software and therefore could be used as a powerful tool
Existing models, even fairly cheap ones, can find security issues and edge-cases reasonably well when applied at scale. Things like buffer overflows aren’t hard to find when you know what you’re looking for and never let your guard down, and an LLM that’s set to constantly scour for them satisfies both criteria. We don’t know if their new model is secretly finding extremely difficult security flaws that older models couldn’t find, but the examples I’ve seen have been fairly conventional.
In other words, my expectation is that Mythos is not discovering AiR-ViBeR—level esoteric data exfiltration techniques. Rather, Anthropic is using their substantial compute resources to conduct a thorough LLM review of major codebases, which any major AI company could perform, in order to build demand for their product and, secondarily, secure positive PR.
Since the release of ChatGPT, at any given time, anyone on the planet with a few bucks could access the current most capable AI model, the SOTA.[4]
Since Mythos, this has no longer been the case and I don’t think it will ever happen again.
I would strongly disagree with the implication here. GPT-2 was infamously guarded in its release. GPT-3 likewise. DALL-E 1 was seen by non-researchers as some manner of crazy secret sauce, complete with a bafflingly uninformative press release about how it worked, until Stable Diffusion became universally available and open source. The thought of independently fine-tuned near-frontier LLMs was unthinkable. It took a long time for the degree of ‘moatlessness’ we currently observe to take shape, and I don’t think a leading company trying to do what leading companies have a long track record of trying to do is sufficient evidence for a sudden reversal of this trend.
I get the impression that what Anthropic is saying isn’t that Mythos is all that much better at finding bugs. It’s that it’s better at converting a hard-to-exploit bug into a working exploit, and even better than that at combining multiple exploits into a practical chain to achieve a goal. That’s the sort of capability where you might expect to get “phase change” behavior.
To your first point, while the post is based on the assumptions that anthropic isn’t lying/exaggerating about Mythos, I still think you can take it at least as Anthropic signaling what they intend to do with a model with potent capabilities. As such I think you’d arrive at the same conclusion.
To your second point, the “Since the release of ChatGPT” is a nontrivial part of the statement, I do agree that it took time for models to be open to the general public but I don’t think the guarding was that severe. For example, I got access (completely random guy) to some OpenAI early research previews, and that was still before the release of ChatGPT. I don’t think its a coincidence AI companies started eagerly asking you to use their models (give them more training data) at around the same time they realized how much more training data they’d need.
Not that it contradicts your thesis really, but I think GPT-4.5 was originally trained for internal use in distillation, with no intention of releasing it to the public (largely due to inference costs). It wasn’t SOTA by the time they released it as a research preview, but it may have been when it was first trained.
The two points I’d push back on:
Existing models, even fairly cheap ones, can find security issues and edge-cases reasonably well when applied at scale. Things like buffer overflows aren’t hard to find when you know what you’re looking for and never let your guard down, and an LLM that’s set to constantly scour for them satisfies both criteria. We don’t know if their new model is secretly finding extremely difficult security flaws that older models couldn’t find, but the examples I’ve seen have been fairly conventional.
In other words, my expectation is that Mythos is not discovering AiR-ViBeR—level esoteric data exfiltration techniques. Rather, Anthropic is using their substantial compute resources to conduct a thorough LLM review of major codebases, which any major AI company could perform, in order to build demand for their product and, secondarily, secure positive PR.
I would strongly disagree with the implication here. GPT-2 was infamously guarded in its release. GPT-3 likewise. DALL-E 1 was seen by non-researchers as some manner of crazy secret sauce, complete with a bafflingly uninformative press release about how it worked, until Stable Diffusion became universally available and open source. The thought of independently fine-tuned near-frontier LLMs was unthinkable. It took a long time for the degree of ‘moatlessness’ we currently observe to take shape, and I don’t think a leading company trying to do what leading companies have a long track record of trying to do is sufficient evidence for a sudden reversal of this trend.
I get the impression that what Anthropic is saying isn’t that Mythos is all that much better at finding bugs. It’s that it’s better at converting a hard-to-exploit bug into a working exploit, and even better than that at combining multiple exploits into a practical chain to achieve a goal. That’s the sort of capability where you might expect to get “phase change” behavior.
To your first point, while the post is based on the assumptions that anthropic isn’t lying/exaggerating about Mythos, I still think you can take it at least as Anthropic signaling what they intend to do with a model with potent capabilities. As such I think you’d arrive at the same conclusion.
To your second point, the “Since the release of ChatGPT” is a nontrivial part of the statement, I do agree that it took time for models to be open to the general public but I don’t think the guarding was that severe. For example, I got access (completely random guy) to some OpenAI early research previews, and that was still before the release of ChatGPT. I don’t think its a coincidence AI companies started eagerly asking you to use their models (give them more training data) at around the same time they realized how much more training data they’d need.
Not that it contradicts your thesis really, but I think GPT-4.5 was originally trained for internal use in distillation, with no intention of releasing it to the public (largely due to inference costs). It wasn’t SOTA by the time they released it as a research preview, but it may have been when it was first trained.