Niall's harsh words about ISO 9000 stirred up proponents and opponents alike. This month, he throws more fuel on the fire.
Last February, I devoted this space to the ISO 9000 standard and its impact on engineering (“A Question of Quality” ). The response from readers amazed me. I mainly wrote the column to get my own frustrations in this area off of my chest, regardless of whether anyone else would listen. To my surprise, I received more responses to that column than all of my other columns combined. Some of them were as long as the column itself. (I've published the responses at www.panelsoft.com/murphyslaw.) Obviously, there is a lot of strong feeling out there about ISO 9000.
Most readers agreed with my opinion that the ISO 9000 approach has done a lot of damage, though there were some robust defenses of the standard. I will address these later. Many of the incidents describing the quality-police-gone-mad would be funny if they were not so sad.
At several companies that have gone out of business, there are some employees who believe that efforts to comply with ISO 9000 contributed significantly to the failure of the business. One reader pointed out that the manager who got his company certified was promoted to vice president-shortly before they went bankrupt. The priorities were wrong; certification was seen as an end in itself. Some organizations believe it's reasonable to pay some kind of price for certification. But if the certification process measures the right parameters, certification should involve no extra work for organizations that already have quality processes in place.
Another theme in many of your e-mails is the number of people who left jobs because bureaucracy prevented them from doing their work properly. This confirms my long-held suspicion that ISO 9000 increases the turnover of engineers. Anyone who has worked on a team where engineers regularly come and go understands the effect it has on productivity. Each new engineer requires a learning period before he becomes useful to the project. One engineer who wrote to me said that ISO 9000 had made a succession of jobs so frustrating that he simply retired.
Many readers echoed my sentiment that design does not lend itself well to being described ahead of time. In some cases, it is the equivalent of the question, “So, Columbus, how long is it going to take you to get to the New World?”
Regulating design is tricky. If the process allows numerous variations on a design path, the description will be too vague. If the process is too restrictive, it may cut off paths that make sense.
Shifting the power
The ISO 9000 process, in many companies, is enforced by a quality department that is a separate entity from the development teams. Without a dedicated group, few companies can achieve or maintain certification. But is it always better to use such a group?
One reader described a positive experience with ISO 9000. He said that it encouraged regression testing, and this made his job considerably easier. When I read his letter, I wanted to know why his experience differed from the many negative stories I heard from other quarters. The crucial difference, I determined, was that he worked in a small development house where the engineer designing the processes was the engineer who executed them. If he designed a bad process, he would be the one to suffer. This is not the case with independent quality departments.
Such departments do not bear the brunt of their own mistakes. Someone else has to work around awkward procedures, or miss deadlines because of paperwork. If you do not suffer from your own mistakes, you are unlikely to learn from them. An independent quality group that polices other engineering groups within the organization has no concrete impetus to improve.
There is a parallel here with testing. To have an effective test team, it is necessary that the test team have some level of independence from the development team, so that the testers can deliver results (report bugs) that may impede the development team's ability to deliver on time. This healthy separation between development and test helps to delay the launch of products that genuinely need more work. Can we apply the same argument to the quality team? I don't think so. First, the quality team attempts to control how the development is done, while the test team is working with the output of the development team. When someone, in any company, attempts to dictate how someone else should do a particular job, they meet some resistance.
More importantly, if developers feel that they have abdicated control, they also feel that they have abdicated some responsibility. If the engineers are not allowed to develop in the way they see fit, they will feel little shame if the product is late or a failure, since they had to play the game by someone else's rules.
The second difference between a test team and a quality team is that in the area of test there is little ambiguity in terms of measured results. For the vast majority of reported bugs, a little investigation can decide, to everyone's satisfaction, whether a problem requires a fix. In the area of quality, there is far more ambiguity. If the quality department decides that extra documentation is required, and the development team feels that it wouldn't add any value (or quality), it is far too easy for the quality department to portray the engineers as lazy and to claim that, without this documentation, they would fail certification. The requirements for ISO 9000 certification are so vague that it is never possible to say whether a particular missing document will be considered a non-conformance (the chances are that one auditor would allow it and another would not). When so many issues come down to “my word against yours,” having one group police another will always lead to unhealthy tensions and resentment. If the product ships late or with too many bugs, the root cause will never be identified as an excess of paperwork caused by the quality department.
Baby with the bathwater
A number of the responses to the February column suggested that I was against documentation, regression testing, and configuration management. I am a huge proponent of these mechanisms. Version-control tools and a bug-tracking database should be the backbone of the quality processes of any development team.
How do we avoid excessive procedures, but keep our quality level high? One of the first principles to apply is to keep the decision-making close to where the real work is being done. Allowing the decisions to be moved outside the development team to a quality group, or external auditors, means that the decisions will not be in tune with the work. Keeping control in the development teams requires good leadership. This is where team leaders should be put to the test more; they are the people who should be driving quality.
I have criticized groups that produce too much documentation. I have worked in teams where there were huge amounts of documentation, but I have also worked in groups with none. Non-documentation is also frustrating. Documents are required for effective decision making, and for effective communication within a team and to groups outside a team.
ISO 9000 encourages documentation for other purposes: to document how work should be done and to document that work has been done. This is important in some cases but in many it is superfluous.
Some companies have a document that describes how design should be performed. This includes a template for the design document itself. If you look at the table of contents of the resulting design documents, they all have, for example, an Introduction, Scope, Audience, References, and Design chapter. The table of contents looks the same on each design document. If a competent engineer were allowed to write the design in any format, the chapters would include headings that related to the actual design, such as, for example, Sensors, Calibration, Control Loop, and Communications.
Check some of your own documents to see if they pass this test. Do they have a table of contents that tells you nothing about the actual contents of the document? This is a trivial complaint, but it is a symptom of a wider problem. It is a case of over-regulation reducing variation when variation actually represents higher quality.
Reds in the bed
A number of the defenders of the ISO 9000 standard claimed that my examples of over-documentation and unwieldy test procedures were simply cases of bad implementation. If procedures are bad, the defenders say, they should simply be changed. The catch is that ISO 9000, by its nature, inhibits changes. Changes require sign-off and justification. When an engineer hits a problem that requires a procedure change, he may have to wait a week to get the appropriate sign-off. If the change is within a large document that is only reviewed and updated occasionally, the wait could be far longer. The nature of research and development is that procedures will not be right the first time. It can take many iterations to get them right.
The “not using it right” argument for ISO 9000 is equivalent to a claim that communism is a better form of government than democratic government and free markets. When confronted with history's overwhelming evidence to the contrary, and with the economic disasters and social abuses that invariably stem from such systems, the communism proponent's position is often that “it wasn't done right,” and that if only those in charge hadn't screwed up, it would produce the utopia promised by the textbooks.
Whether the theory is right or wrong is irrelevant when empirical evidence tells us that, in practice, many communist governments become corrupt. So it is with ISO 9000. If companies are using it incorrectly in the majority of cases, it is doing more harm than good.
Controlling the process
A case could be made that companies need to police engineering activity, and they need something like ISO 9000 certification to give their position some teeth, so that they can ensure procedures are followed. How else do we stop lazy engineers from skipping some part of good engineering practice?
I am afraid this tends to backfire. In my experience, lazy engineers love ISO 9000 and everything that goes with it. Lazy engineers need proof that they have done work, and it is far easier to produce a document than a working piece of software. In an ISO 9000 environment, evidence of work performed is measured in documents, instead of products produced.
It's hard for employees to resist when a company decides to go down the ISO 9000 road. I chose to write about this topic so that some engineers would learn to recognize the difference between ISO 9000 certification and quality. Ideally, you should have quality without ISO 9000 certification, but if you do seek ISO 9000 certification, try to avoid reducing the quality that's already there. esp
Niall Murphy has been writing software for user interfaces and medical systems for ten years. He is the author of Front Panel: Designing Software for Embedded User Interfaces. Murphy's training and consulting business is based in Galway, Ireland. He welcomes feedback and can be reached at .
Thanks to Matt and Florrie for help on this column.