I agree there’s a big challenge coming, but we also have an opportunity to improve cybersecurity, as I’ve written about elsewhere.
Briefly: your narrative presents computer systems like biological systems, implicitly created by an evolutionary process we have no control over, so of course obscure vulnerabilities are inevitable. However, we have the chance to integrate new vulnerability-checking into the software-development lifecycle. Even better, we can lean into formal verification to prove mathematically that systems are secure, so future models can’t find bugs of certain kinds. They both sound like hard goals from a year-2020 perspective, but generative AI is accelerating both dramatically. In fact, I recently argued here that the time has nearly come to routinely throw away all old code and regenerate from scratch following new wisdom in security and other requirements dimensions—because the cost of reliable software engineering should drop so dramatically.
I agree there’s a big challenge coming, but we also have an opportunity to improve cybersecurity, as I’ve written about elsewhere.
Briefly: your narrative presents computer systems like biological systems, implicitly created by an evolutionary process we have no control over, so of course obscure vulnerabilities are inevitable. However, we have the chance to integrate new vulnerability-checking into the software-development lifecycle. Even better, we can lean into formal verification to prove mathematically that systems are secure, so future models can’t find bugs of certain kinds. They both sound like hard goals from a year-2020 perspective, but generative AI is accelerating both dramatically. In fact, I recently argued here that the time has nearly come to routinely throw away all old code and regenerate from scratch following new wisdom in security and other requirements dimensions—because the cost of reliable software engineering should drop so dramatically.