I found Connor’s text very helpful and illuminating!
…But yeah, I agree about sloppy wording.
Instead of “you want to prove even minimal things about it” I think he should have said “you want to prove certain important things about it”. Or actually, he could have even said “you want to have an informed guess about certain important things about it”. Maybe a better example would be “it doesn’t contain a backdoor”—it’s trivial if you’re writing the code yourself, hard for a binary blob you find on the internet. Having access to someone else’s source code helps but is not foolproof, especially at scale, (e.g.).
Well, hmm, I guess it’s tautological that if you’re writing your own code, you can reliably not put backdoors in it. There’s no such thing as an “accidental backdoor”. If it’s accidental then you would call it a “security flaw” instead. But speaking of which, it’s also true that security flaws are much easier to detect or rule out if you’re writing the code yourself than if you find a binary blob on the internet.
Or the halting problem: it’s super-easy to write code that will definitely halt, but there are at least some binary blobs for which it is impossible in practice to know or even guess whether it will halt or not.
…Also, while we’re nitpicking, Connor wrote “175B FP numbers generated by unknown slop methods”. I would have instead said “175B FP numbers generated by gradient descent on internet text” or “175B FP numbers generated by a learning algorithm with no known security-related invariants” or something. (“Unknown” is false and “slop” seems to be just throwing shade (in this context).)
My impression is that if you walk of to a security researcher and say “hey what do you call the kind of thing where for example you’re not properly escaping strings embedded in other strings, like SQL injection?”, they probably wouldn’t say “oh that thing is called an accidental backdoor”, rather they would say “oh that thing is called a security vulnerability”.
(This is purely a terminology discussion, I’m sure we agree about how SQL injection works.)
I guess “backdoor” suggests access being exclusive to person who planted it, while “vulnerability” is something exploitable by everyone? Also, after thinking a bit more about it, I think you’re right that “backdoor” implies some intentionality, and perhaps accidental backdoor is an oxymoron.
I found Connor’s text very helpful and illuminating!
…But yeah, I agree about sloppy wording.
Instead of “you want to prove even minimal things about it” I think he should have said “you want to prove certain important things about it”. Or actually, he could have even said “you want to have an informed guess about certain important things about it”. Maybe a better example would be “it doesn’t contain a backdoor”—it’s trivial if you’re writing the code yourself, hard for a binary blob you find on the internet. Having access to someone else’s source code helps but is not foolproof, especially at scale, (e.g.).
Well, hmm, I guess it’s tautological that if you’re writing your own code, you can reliably not put backdoors in it. There’s no such thing as an “accidental backdoor”. If it’s accidental then you would call it a “security flaw” instead. But speaking of which, it’s also true that security flaws are much easier to detect or rule out if you’re writing the code yourself than if you find a binary blob on the internet.
Or the halting problem: it’s super-easy to write code that will definitely halt, but there are at least some binary blobs for which it is impossible in practice to know or even guess whether it will halt or not.
…Also, while we’re nitpicking, Connor wrote “175B FP numbers generated by unknown slop methods”. I would have instead said “175B FP numbers generated by gradient descent on internet text” or “175B FP numbers generated by a learning algorithm with no known security-related invariants” or something. (“Unknown” is false and “slop” seems to be just throwing shade (in this context).)
There is such thing as accidental backdoor: not properly escaping strings embeded in other strings, like SQL injection, or prompt injection
My impression is that if you walk of to a security researcher and say “hey what do you call the kind of thing where for example you’re not properly escaping strings embedded in other strings, like SQL injection?”, they probably wouldn’t say “oh that thing is called an accidental backdoor”, rather they would say “oh that thing is called a security vulnerability”.
(This is purely a terminology discussion, I’m sure we agree about how SQL injection works.)
I guess “backdoor” suggests access being exclusive to person who planted it, while “vulnerability” is something exploitable by everyone? Also, after thinking a bit more about it, I think you’re right that “backdoor” implies some intentionality, and perhaps accidental backdoor is an oxymoron.