And one thing that bugs me about these “studies” is precisely that they are all too vague as to exactly how (or even whether) they did this.
It’s difficult enough tracing faults back to the source code responsible for them: as every programmer knows, a fault may have multiple “sufficient causes”, several places in the code that you could change to fix the behavior. It’s not always the case that the line of code you change to fix the problematic behavior is the same line of code that introduced a defect. There is often a “bubble under the wallpaper” effect, a conservation of total bugginess, such that a “fix” in one place causes some other functional behavior to break.
It’s even more difficult to trace a defect in code back to a design decision or a design document: the causal pathways pass through one or more human brains (unless you’re using code generation from design documents, in which case the latter count as high-level “code” in my view). Human brains are notoriously non-auditable, they are both extremely tolerant of ambiguity and extremely vulnerable to it, a mixture which causes no end of frustration in software development.
The same arguments apply to “bugs in requirements”, in spades. What exactly does a bug in requirements consist of, and is it even the same beast as a bug in the code? Requirements document have no behavior—there is no way to “test” them, and in one sense requirements are always correct. How could they be incorrect—if we define “requirements” as “whatever the actual code is going to be judged against”? If we don’t define them that way, what is it that serves as a reference against which the requirements document will be judged correct or not, and why don’t we call that the requirements?
Any study of these phenomena needs to put forth what assumptions it operates under, and I have yet to see one empirical study which even attempted such a discussion.
You don’t know that a bug has been introduced when it was introduced, but once you fix it you can trace back to see when it was introduced.
You can often trace back to see when it was introduced prior to fixing it but after identifying the symptom. That’s actually a useful debugging tactic.
You don’t know that a bug has been introduced when it was introduced, but once you fix it you can trace back to see when it was introduced.
And one thing that bugs me about these “studies” is precisely that they are all too vague as to exactly how (or even whether) they did this.
It’s difficult enough tracing faults back to the source code responsible for them: as every programmer knows, a fault may have multiple “sufficient causes”, several places in the code that you could change to fix the behavior. It’s not always the case that the line of code you change to fix the problematic behavior is the same line of code that introduced a defect. There is often a “bubble under the wallpaper” effect, a conservation of total bugginess, such that a “fix” in one place causes some other functional behavior to break.
It’s even more difficult to trace a defect in code back to a design decision or a design document: the causal pathways pass through one or more human brains (unless you’re using code generation from design documents, in which case the latter count as high-level “code” in my view). Human brains are notoriously non-auditable, they are both extremely tolerant of ambiguity and extremely vulnerable to it, a mixture which causes no end of frustration in software development.
The same arguments apply to “bugs in requirements”, in spades. What exactly does a bug in requirements consist of, and is it even the same beast as a bug in the code? Requirements document have no behavior—there is no way to “test” them, and in one sense requirements are always correct. How could they be incorrect—if we define “requirements” as “whatever the actual code is going to be judged against”? If we don’t define them that way, what is it that serves as a reference against which the requirements document will be judged correct or not, and why don’t we call that the requirements?
Any study of these phenomena needs to put forth what assumptions it operates under, and I have yet to see one empirical study which even attempted such a discussion.
You can often trace back to see when it was introduced prior to fixing it but after identifying the symptom. That’s actually a useful debugging tactic.