How I test software, again and again
This is the continuation of my article, "How I Test Software" (July/August, page 11). We have a lot to cover this time, so let's get right to it. I'll begin with some topics that don't fit neatly into the mainstream flow.
Test whose software?
As with my previous article, "How I Test Software", I need to make clear that the chosen pronoun in the title is the right one. This is not a tutorial on software testing methodologies. I'm describing the way I test my software.
Does this mean that I only write software solo and for my own use? Not at all. I've worked on many large projects. But when I do, I try to carve out a piece that I can get my arms around and develop either alone or with one or two like-minded compadres. I've said before that I usually find myself working with folks that don't develop software the same way I do. I used to think I was all alone out here, but you readers disabused me of that notion. Even so, my chances of finding a whole project team who work the way I do are slim and none. Which is why I try to carve out a one-person-sized piece.
What's that you say? Your project is too complicated to be divided up? Have you never heard of modular programming? Information hiding? Refactoring? Show me a project whose software can't be divided up, and I'll show you a project that's doomed to failure.
The classical waterfall method has a box called "Code and Unit Test." If it makes you feel better, you can put my kind of testing into that pigeonhole. But it's more than that, really. There's a good reason why I want to test my software, my way: my standards are higher than "theirs."
Remember my discussion about quality? To some folks, the term "quality" has come to mean "just good enough to keep from getting sued." I expect more than that from my software. I don't think it's enough that it works well enough for the company to get their fee. I think it should not only run, but give the right answer.
I'm funny that way. It doesn't always endear me to my managers, who sometimes just want to ship anything, whether it works or not. Sometimes I want to keep testing when they wish I wouldn't. But regardless of the quality of the rest of the software, I try very hard to make mine bulletproof. Call it pride if you like or even hubris. I'm still going to test thoroughly.
Oddly enough, the reason I spend so much time testing is, I'm lazy. I truly hate to debug; I hate to single-step; I hate to run hand checks. But there's something I hate even more, and that's having to come back again, later, and do it all over again. The reason I test so thoroughly is, I don't ever want to come back this way again.
All my old software engineering textbooks used to say that the most effective method to reduce software errors was by desk checking. Personally, I've never found it to be very effective.
I assume you know the theory of desk checking. When a given programmer is working on a block of code, he may have been looking at the code so long, he doesn't see the obvious errors staring him in the face. Forest for the trees, and all that stuff. A fresh pair of eyes might see those errors that the programmer has become blind to.
Maybe it's so, but I find that the folks that I'd trust to desk check my code, and do it diligently, are much too much in demand to have time to do it. And what good is it to lean on a lesser light? If I have to spend all day explaining the code to him, what good is that?
But there's another, even more obvious reason to skip desk checking: it's become an anachronism. Desk checking may have been an effective strategy in one of those projects where computer time was so precious, no one was allowed to actually compile their code. But it makes no sense at all, today.
Face it, unless the checker is a savant who can solve complicated math equations in his head, he's not going to know whether the math is being done right or not. At best, he can only find syntax errors and typos. But today, there's a much more effective way to find those errors: let the compiler find them for you. When it does, you'll find yourself back in the editor with the cursor blinking at the location of the error. And it will do it really fast. With tools like that, who needs the fellow in the cubie next door?