Stanislav Fort

Karma: 482

Stanislav Fort 13 Aug 2026 19:26 UTC
1 point
0
in reply to: Muhan Luo’s comment on: Are Mythos’ Cyber Capabilities Overstated? - Yes and No
That’s incorrect. Here’s the situation chronologically:
1. AISLE has been scanning cURL for some time since mid 2025. This revealed a number of zero-days that got fixed and assigned CVEs.
2. In May, cURL ran Mythos, which found only a single real vulnerability of low severity. It got fixed, and assigned a CVE.
3. after that, on the fixed version, we ran AISLE, discovered 6 additional zero-days that Mythos missed, and we got 6 CVEs for them. That’s what https://aisle.com/blog/aisle-discovers-6-new-cves-in-curl-including-the-oldest-issue-ever-reported is talking about, and that’s what the literally author of cURL is writing about here: https://mastodon.social/@bagder/116807317163361428
We also got “one shot”, in the same sense that Mythos got “one shot”. Neither is literally a single model invocation since both systems run multiple parallel analyses across the codebase.

So continuous scanning does not explain this result. On curl = one of the most hardened codebases in existence, AISLE found six confirmed zero-days after Mythos’s scan and subsequent hardening. Mythos found one. On this target, AISLE clearly outperformed Mythos at zero-day discovery. You got the timeline wrong in your previous comment.

Stanislav Fort 4 Aug 2026 19:17 UTC
1 point
0
on: Are Mythos’ Cyber Capabilities Overstated? - Yes and No

So a low finding count on cURL may say more about cURL’s low attack surface and robust security practices than about Mythos’ capabilities.

Came here to say that you’re wrong on this. While Mythos found a single zero-day that got a CVE in curl + 1 sub-CVE bug + 3 false positive, AISLE actually discovered 6 zero-days that received CVEs after Mythos was done with the codebase and its detections fixed ⇒ Mythos missed these 6 legitimate zero-days: https://mastodon.social/@bagder/116807317163361428

Stanislav Fort 23 Jun 2026 8:47 UTC
1 point
0
on: Are Mythos’ Cyber Capabilities Overstated? - Yes and No
I think you might be conflating AISLE’s toy experiments with tiny models vs our actual zero-day detection performance on some of the most hardened codebases out there (e.g. curl, FreeBSD, OpenSSL). On publicly available zero-day discovery evidence, AISLE matches and sometimes even surpasses Mythos. I compiled some of the CVE-level evidence here: https://stanislavfort.substack.com/p/mythos-at-home-and-its-called-aisle

Stanislav Fort 21 May 2026 7:21 UTC
1 point
0
in reply to: Jessica Rumbelow’s comment on: An Introduction to Exemplar Partitioning for Mechanistic Interpretability
Not all all, the distance is in whatever layer’s activation space you are, where ±middle layers work the best. It’s also not e.g. L2, because we were using a “learned” covariance matrix and getting something like a Mahalanobis distance.

Stanislav Fort 16 May 2026 17:25 UTC
5 points
0
on: An Introduction to Exemplar Partitioning for Mechanistic Interpretability
Interesting! I used a similar technique for a very particular application, namely detecting harmful inputs into LLMs in Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Section 5.4 (Applying Out-of-Distribution Detection to Reject Strange or Harmful Requests).

Stanislav Fort 2 May 2026 5:27 UTC
2 points
0
on: Sanity-checking “Incompressible Knowledge Probes”
Quick thought: I wonder if you could still be overestimating parameter counts for models that would be made by distilling a larger teacher down to a smaller student. Any OSS ones you could test this hypothesis on?

Stanislav Fort 9 Feb 2026 12:40 UTC
7 points
0
in reply to: Rasool’s comment on: AI found 12 of 12 OpenSSL zero-days (while curl cancelled its bug bounty)
Thanks for flagging this, Rasool. I’ve been following the Anthropic announcement closely. It’s genuinely exciting to see a frontier lab validate that AI can find real vulnerabilities in real software at scale. The more serious players in this space, the better, since it is a genuinely large problem ⇒ far larger than any single team.

We’ve been doing this work operationally since mid/late-2025, and our experience has been that the hardest part isn’t finding some bugs but rather the hardest bugs in the most audited codebases. Earning the trust of maintainers and closing the full loop from discovery through patch acceptance is also very challenging. That’s where most of the difficulty (and most of the value) lives.

I wrote up a practitioner’s perspective on what we’ve learned, how our results compare, and what the real challenges ahead look like from our vantage point: What AI Security Research Looks Like When It Works

Stanislav Fort 8 Feb 2026 14:52 UTC
4 points
0
in reply to: Quinn’s comment on: AI found 12 of 12 OpenSSL zero-days (while curl cancelled its bug bounty)

i don’t know for sure that there is zero “formal methods” in the pipeline

in discovering the 12 OpenSSL zero-day vulnerabilities, we haven’t used any formal methods. since then, we incorporated some. the (discovery --> CVE assigned --> CVE made public) pipeline is a very lagging indicator and the OpenSSL results are reflective of the state of the AISLE system approximately mid-fall 2025, prior to our use of formal methods

AI found 12 of 12 OpenSSL zero-days (while curl cancelled its bug bounty)

Stanislav Fort27 Jan 2026 20:21 UTC

359 points

25 comments8 min readLW link

Stanislav Fort 3 Nov 2025 17:26 UTC
4 points
0
in reply to: peterbarnett’s comment on: AISLE discovered three new OpenSSL vulnerabilities
Short answer: these aren’t Heartbleed-class, but they’re absolutely worth patching.

Two signals: (i) OpenSSL itself minted CVEs for them. This is non-trivial given its conservative posture, and (ii) fixes were backported across supported branches (3.5.4 / 3.4.3 / 3.3.5 / 3.2.6, with distro backports).

For context, per OpenSSL’s own vulnerability index as of today (3 Nov 2025), there were 4 CVEs in 2025 YTD (CVE-2025-), 9 in 2024 (CVE-2024-), 18 in 2023 (CVE-2023-), 15 in 2022 (CVE-2022-). Getting any CVE there is hard. “Low/Medium” here mostly reflects narrow preconditions and prevalence within typical OpenSSL usage, not that the primitives themselves are trivial. The score (called CVSS) compresses likelihood and impact into one scalar.

Stanislav Fort 3 Nov 2025 17:12 UTC
11 points
0
in reply to: Dean Valentine’s comment on: AISLE discovered three new OpenSSL vulnerabilities
Appreciate the pushback and your perspective. Two anchoring facts:
1. OpenSSL minted and published these CVEs (not us). They’re very conservative. Getting any CVE through their process is non-trivial. In 2025 we reported several issues. Some received CVEs, others were fixed without CVEs, which is normal under OpenSSL’s security posture.
2. On your “AI vs human experts” point: the findings came from a fully autonomous analysis pipeline. We then manually verified and coordinated disclosure with maintainers. The takeaway: our stack surfaced previously unknown, CVE-worthy bugs in OpenSSL’s hardened codebase. That’s hard to do by hand at scale.

Solving adversarial attacks in computer vision as a baby version of general AI alignment

Stanislav Fort29 Aug 2024 17:17 UTC

101 points

10 comments7 min readLW link