This post answers the question: Is AI smart enough to largely independently, and without any special structure find vulnerabilities?
Personally I think the following question is more important (and interesting): Can we use AI to find vulnerabilities autonomously?
This seems to be something AIxCC tries to answer.
I recently saw a Hank Green video (https://youtu.be/DpGU8NARX-s?t=558) where he gives a definition of pseudoscience that I thought was pretty good.
The basic idea is that science is the process by which we create evidence for something, and then have that evidence be challenged. Doing this repeatedly helps us notice that some structures are pretty good for creating good evidence (ie statistics, double blind trials, falsifiability, etc.). Pseudoscience is using these same structures to make what you’re doing look like science, without actually interacting with the “being challenged” part.