Thanks @noahzuniga ! I think the continuum of AI capability very, very roughly looks like:
AI cannot even help with this task.
AI can help an expert human with this task.
AI can mostly do this task autonomously, with some light steering and feedback from a human.
AI can do this task autonomously provided the right harness design.
AI can basically one-shot this task.
My blog post, IMO, confirmed we are not at stage 5 yet. I am very confident we are at least at stage 2. I am not certain where in the range [2, 4] we fall currently, though. I plan to do more work to further refine my assessment.
This post answers the question: Is AI smart enough to largely independently, and without any special structure find vulnerabilities?
Personally I think the following question is more important (and interesting): Can we use AI to find vulnerabilities autonomously?
This seems to be something AIxCC tries to answer.
Thanks @noahzuniga ! I think the continuum of AI capability very, very roughly looks like:
AI cannot even help with this task.
AI can help an expert human with this task.
AI can mostly do this task autonomously, with some light steering and feedback from a human.
AI can do this task autonomously provided the right harness design.
AI can basically one-shot this task.
My blog post, IMO, confirmed we are not at stage 5 yet. I am very confident we are at least at stage 2. I am not certain where in the range
[2, 4]we fall currently, though. I plan to do more work to further refine my assessment.