But software defects are nothing like defects in physical materials. The layers of which software is built are all equally accessible
I don’t think this is quite true. For instance, a few years ago, I traced a bug in my application down to an issue in how the Java Virtual Machine does JIT compiling, which caused subtle differences in a numerical algorithm between when the application started up, and when it had warmed up enough that certain functions were JIT compiled. Almost certainly, the correct fix would have been to correct the JVM so that the results were exactly the same in all cases.
But, of course, the JVM was nowhere near as “accessible” as the library containing the bug—almost everyone relies on a prebuilt version of the JVM, and it is rather difficult to build. Also, it’s written in a different and less friendly language: C++. Of course, this assumes that we are using a free/open source JVM (as it happens, we are); the situation would be even worse if we had to rely on a proprietary VM. And it assumes that all of our users would have been willing to use a custom JVM until a fixed version of the mainline JVM were released.
Another possibility would have been to add a compile-time option to the library containing that algorithm, so that that particular function would either always be JIT-compiled, or would never be. That’s pretty straightforward—as it happens, a different division of my company employs some of that library’s authors. But the authors didn’t consider it a worthwhile thing to do. So now we could maintain a fork of the library forever, or we could fix the bug somewhere else. Again, of course, this relied on the library being open; with a typical proprietary library, there would have been no recourse.
Needless to say, the top layer, the application, was the easiest thing to change, and so that’s what changed.
Neither lower-level would have negatively impacted other library users (well, maybe turning off JIT on this function might have, but turning it always-on wouldn’t). So I do think there is, in some sense, a difference in accessibility between layers which is not just caused by the interdependence problem. We genuinely do treat lower layers as foundational, because it makes it easier to develop, distribute, and collaborate on software. So I’m not sure that a construction analogy is entirely inappropriate here.
So I’m not sure that a construction analogy is entirely inappropriate here.
Good observations—but note that these criteria for “accessible” (and the consequences you discuss) are socio-political in nature, rather than physical: the JVM is the result of someone else’s analysis, design, programming, testing etc. - and your decision to use it is not part of the software life-cycle depicted in the diagram.
A theory which attempted to account for such differences would find its ontology invaded with notions of copyright, software licensing, organizational divisions and the like—the SDLC would no longer be sufficient.
Some of them are socio-political, but I think others are intellectual. That is, I understand the internals of my program well, the library somewhat, and the JVM barely at all. And this would probably be close to accurate even if I had written the whole stack myself, since I would have written the JVM the longest time ago, the library more recently, and the program last. Buildings are built of stuff; programs are built of thoughts. That some information is more accessible because you have used it more recently is a fact about the brain rather than about software. But brains (and organizations) are all we have to build software with. So I think any methodology that does not account for these things must be incomplete.
Buildings are built of stuff; programs are built of thoughts. That some information is more accessible because you have used it more recently is a fact about the brain rather than about software.
There you have, in a nutshell, the problem with software engineering as a formal discipline: its stubborn refusal to admit the above, in the face of determined pushes to do so from the likes of Bill Curtis (who’s been refused a Wikipedia entry because he’s less notable than any number of porn stars) or Jerry Weinberg.
Dijkstra’s view was that the limitations of the human mind are precisely the reason that software must be treated as mathematics and developed with mathematical rigour.
That depends on where the mathematics is done. Dijkstra’s and Hoare’s vision of programmers proving their own code correct with pencil and paper is unworkable. People cannot reliably do any sort of formal manipulation on paper, not even something as simple as making an exact copy of a document. The method can be exhibited on paper for toy examples, but everything works for toy examples. So what to do?
Compare the method of writing machine code by developing a method of translating a higher-level language into machine code. This can be exhibited on paper for toy examples, but of course that is only for didactic purposes, and one writes a compiling program to actually do that work in production. This reduces the work of writing programs in machine code to the task of writing just one program in machine code, the compiler, and by bootstrapping techniques that can be reduced even further. The amount of mathematics carried out on the human side of the interface is greatly reduced.
Similarly, proving things about programs has to be done automatically, or there’s no point. We have to prove things about programs, because computing hardware and software is mathematics, whether we wish it to be or not. Software engineering is precisely the problem of how human beings, who cannot reliably do mathematics, can reliably instruct a mathematical machine to do what we want it to with mathematical reliability.
I don’t have any solutions, it just seems to me that this is how to describe the problem. How do we interface thought-stuff with machine-stuff?
In this case, it’s not clear that the compiler was really wrong. The results of a floating point calculation differed by a tiny amount, and it’s possible that either was acceptable (I don’t know how strict Java is about its floating point rules). The problem was that I was using the result as a hash key.
But later, I was able to make the JVM reliably dump core (in different memory locations each time). Unfortunately, it was only under extremely heavy load, and I was never able to build a reduced test case.
Compilers do get things wrong. You may be interested in John Regehr’s blog; he’s essentially throwing cleverly-chosen “random” input at C compilers (“fuzz-testing”). The results are similar to those for other programs that have never been fuzzed, i.e. embarrassing.
Well, your prior should be pretty high that it’s your fault, unless you also wrote the compiler :)
If you can do experiments to prove that there’s a compiler bug, you learn something. If you jump straight to the compiler bug explanation instead of looking for the bug in your own code, you are resisting education, and the probability that all you are doing is delaying the lesson is the probability that the compiler is working correctly. This should be >>50% of the time or you need a better compiler!
The difference here is not so much in where you guess the bug is, as in whether you do the experiment.
A very effective experiment is to take your program and chop out everything irrelevant until you have a short piece of code which demonstrates the bug. At this point, if it is a compiler bug, you have dense information to hand the compiler author; if it isn’t a compiler bug, you’re in a much better position to understand what’s wrong with your code.
However, one is often reluctant to apply this technique until one suspects a compiler bug, because it seems like a lot of work. And it is — but often less work than continuing to examine the bug with less radical tools, given that you’re in the position where the notion of compiler bugs crosses your mind.
I don’t think this is quite true. For instance, a few years ago, I traced a bug in my application down to an issue in how the Java Virtual Machine does JIT compiling, which caused subtle differences in a numerical algorithm between when the application started up, and when it had warmed up enough that certain functions were JIT compiled. Almost certainly, the correct fix would have been to correct the JVM so that the results were exactly the same in all cases.
But, of course, the JVM was nowhere near as “accessible” as the library containing the bug—almost everyone relies on a prebuilt version of the JVM, and it is rather difficult to build. Also, it’s written in a different and less friendly language: C++. Of course, this assumes that we are using a free/open source JVM (as it happens, we are); the situation would be even worse if we had to rely on a proprietary VM. And it assumes that all of our users would have been willing to use a custom JVM until a fixed version of the mainline JVM were released.
Another possibility would have been to add a compile-time option to the library containing that algorithm, so that that particular function would either always be JIT-compiled, or would never be. That’s pretty straightforward—as it happens, a different division of my company employs some of that library’s authors. But the authors didn’t consider it a worthwhile thing to do. So now we could maintain a fork of the library forever, or we could fix the bug somewhere else. Again, of course, this relied on the library being open; with a typical proprietary library, there would have been no recourse.
Needless to say, the top layer, the application, was the easiest thing to change, and so that’s what changed.
Neither lower-level would have negatively impacted other library users (well, maybe turning off JIT on this function might have, but turning it always-on wouldn’t). So I do think there is, in some sense, a difference in accessibility between layers which is not just caused by the interdependence problem. We genuinely do treat lower layers as foundational, because it makes it easier to develop, distribute, and collaborate on software. So I’m not sure that a construction analogy is entirely inappropriate here.
Good observations—but note that these criteria for “accessible” (and the consequences you discuss) are socio-political in nature, rather than physical: the JVM is the result of someone else’s analysis, design, programming, testing etc. - and your decision to use it is not part of the software life-cycle depicted in the diagram.
A theory which attempted to account for such differences would find its ontology invaded with notions of copyright, software licensing, organizational divisions and the like—the SDLC would no longer be sufficient.
Some of them are socio-political, but I think others are intellectual. That is, I understand the internals of my program well, the library somewhat, and the JVM barely at all. And this would probably be close to accurate even if I had written the whole stack myself, since I would have written the JVM the longest time ago, the library more recently, and the program last. Buildings are built of stuff; programs are built of thoughts. That some information is more accessible because you have used it more recently is a fact about the brain rather than about software. But brains (and organizations) are all we have to build software with. So I think any methodology that does not account for these things must be incomplete.
There you have, in a nutshell, the problem with software engineering as a formal discipline: its stubborn refusal to admit the above, in the face of determined pushes to do so from the likes of Bill Curtis (who’s been refused a Wikipedia entry because he’s less notable than any number of porn stars) or Jerry Weinberg.
Dijkstra’s view was that the limitations of the human mind are precisely the reason that software must be treated as mathematics and developed with mathematical rigour.
Note that this is the exact opposite of using the native architecture.
That depends on where the mathematics is done. Dijkstra’s and Hoare’s vision of programmers proving their own code correct with pencil and paper is unworkable. People cannot reliably do any sort of formal manipulation on paper, not even something as simple as making an exact copy of a document. The method can be exhibited on paper for toy examples, but everything works for toy examples. So what to do?
Compare the method of writing machine code by developing a method of translating a higher-level language into machine code. This can be exhibited on paper for toy examples, but of course that is only for didactic purposes, and one writes a compiling program to actually do that work in production. This reduces the work of writing programs in machine code to the task of writing just one program in machine code, the compiler, and by bootstrapping techniques that can be reduced even further. The amount of mathematics carried out on the human side of the interface is greatly reduced.
Similarly, proving things about programs has to be done automatically, or there’s no point. We have to prove things about programs, because computing hardware and software is mathematics, whether we wish it to be or not. Software engineering is precisely the problem of how human beings, who cannot reliably do mathematics, can reliably instruct a mathematical machine to do what we want it to with mathematical reliability.
I don’t have any solutions, it just seems to me that this is how to describe the problem. How do we interface thought-stuff with machine-stuff?
Wow. An actual exception to “The compiler is never wrong!”
In this case, it’s not clear that the compiler was really wrong. The results of a floating point calculation differed by a tiny amount, and it’s possible that either was acceptable (I don’t know how strict Java is about its floating point rules). The problem was that I was using the result as a hash key.
But later, I was able to make the JVM reliably dump core (in different memory locations each time). Unfortunately, it was only under extremely heavy load, and I was never able to build a reduced test case.
Compilers do get things wrong. You may be interested in John Regehr’s blog; he’s essentially throwing cleverly-chosen “random” input at C compilers (“fuzz-testing”). The results are similar to those for other programs that have never been fuzzed, i.e. embarrassing.
And yet, in practice, when something is wrong with your code, it’s always your own fault.
Well, your prior should be pretty high that it’s your fault, unless you also wrote the compiler :)
If you can do experiments to prove that there’s a compiler bug, you learn something. If you jump straight to the compiler bug explanation instead of looking for the bug in your own code, you are resisting education, and the probability that all you are doing is delaying the lesson is the probability that the compiler is working correctly. This should be >>50% of the time or you need a better compiler!
The difference here is not so much in where you guess the bug is, as in whether you do the experiment.
A very effective experiment is to take your program and chop out everything irrelevant until you have a short piece of code which demonstrates the bug. At this point, if it is a compiler bug, you have dense information to hand the compiler author; if it isn’t a compiler bug, you’re in a much better position to understand what’s wrong with your code.
However, one is often reluctant to apply this technique until one suspects a compiler bug, because it seems like a lot of work. And it is — but often less work than continuing to examine the bug with less radical tools, given that you’re in the position where the notion of compiler bugs crosses your mind.