More on Certification -

More on Certification


In the August issue of the Communications of the ACM (“Why the Software Industry Needs a Good Ghostbuster”) Jeanette Nasem Morgan argues that software developers should be certified before being allowed to write commercial code. She complains that most other professionals already have certification processes in place… so why don’t we? That muddled reasoning hardly, in my view, justifies a wholesale revamping of the engineering profession.

She worries that the tsunami of well-known software failures means we need some sort of Generally Accepted Software Engineering Practices, mirroring those adopted by CPAs and other professions.

But then she writes: “Much like the teeming, seething, gelatinous evil surging under the streets of a Gotham City-like New York in the film Ghostbusters, the software engineering industry has become a labyrinth of varied practices and procedures. There is little consensus but many points of view on how a software consultant or professional might analyze a client’s requirements.”

And that’s the problem.

There is no body of generally accepted software engineering practices. IEEE pushes the Software Engineering Body of Knowledge (SWEBOK – find it at, a summary of practices that simply don’t align with what most developers actually do. The IEEE offers the Certified Software Development Professional (CSDP) certification which is mostly based on practices outlined in the SWEBOK.

The Software Engineering Institute’s Capability Maturity Model is a well-known, lightly-used process that can yield good code. My informal survey of the embedded space shows that under 2% of companies are CMM certified at any level. And many certified outfits don’t actually practice what the CMM preaches.

The SEI also pushes the Team Software Process (TSP) and Personal Software Process (PSP), nice ideas only rarely employed in the embedded space.

Today there’s a big backlash against heavyweight processes like the CMM. Now it’s all about being agile, and an alphabet soup of agile methods have sprung up. Some of the practices are much like those embraced by the SWEBOK and by the CMM. Others are orthogonal. There’s little question agile methods are gaining acceptance, yet they’re barely mentioned in the SWEBOK and are ignored by the CMM, PSP and TSP.

Other standards attempt to control development in different ways. MISRA constrains the use of C in motor vehicle applications. The FAA, FDA, and Federal Elections Commission all have very different standards that all aim to insure safe software.

We have many – too many – standards and development methodologies. Though I doubt there will ever be a one-size-fits-all approach the plethora that exists today shows the immature state of this very young field. One reason certification works for CPAs is that there’s really only one way to balance the books. Learn that, pass a test, and you’re judged competent.

I have little doubt that software certification is coming. This is the only industry where we continue to get away with shipping products with known defects. Sooner or later people will stop smoking and all the asbestos will be safely buried somewhere, so the lawyers, looking desperately for a new source of revenue, will go after us. Insurers fed up with litigation payouts will demand that developers be certified or licensed in some manner.

But let’s hope that doesn’t happen till we can agree on the “right” way (or ways) to build software.

What do you think?

Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at . His website is .

Certification of software engineers won't solve any problem. People areas good as their environment expects/demands. If someone is working ina sweat-shop that simply wants to crank out code at “X” lines per day,then no amount of training or certification will produce good code.

On the other hand, if the environment demands well-designed code, ifthe code gets regular reviews by competent peers, proper testing, andmanagement willing to invest the time, then every software engineer willbegin producing much better quality code.

Over the last 23+ years writing professional software, I've noticed thatsome people are simply better, some not, at writing code. I've alsonoticed that experience and diversification makes a big difference.Someone who's worked in a lot of different environments gets moreexposure to good and bad practices, and hopefully picks up goodskills while remembering horrific experiences and trying not to do thesame thing.

Certification is simply a way for the lesser engineers to “look as good”as the more experienced people. Certification has nothing to do withqualifications nor skill. We might as well use shoe size.

About 15 years ago, New Jersey attempted to enact a law that requiredall software engineers to be licensed/certified. How did this come about?Some out-of-work software person got the ear of a politician and moanedthat it's not fair that he was out of work despite lots of questionabletraining classes. The politician, always wanted to help (or at least get avote) decided that requiring certification would fix this great socialinjustice. The company I was consulting for at the time was filled withreally top-notch people… the company fought this law aggressively, andit was eventually defeated. Why did the company fight it? They had agreat group of top-notch people with many years experience, lots ofdesire to solve customer problems, enthusiasm to take on new projects,self-motivated, and self-training. If the certification law passed, all oftheirexcellent engineers would have to go back for training; some would takeseveral years since they didn't all have college degrees. How wouldcertification improve the excellent group they already had? It wouldn't!

Excellent Engineers are good without “certification”, but certificationallows complete losers to suddenly appear to be much better than theyreally are.

Just my two cents.

– Bob Applegate

There were a couple of of statements in your latest column about certification that were interesting. The statement that the SWEBOK, for example, doesn't align with what most developers actually do. The other statement that only 2% of companies are CMM certified at any level and that certified outfits don't actually practice what the CMM preaches.

I don't think that these are valid arguments that the SWEBOK or the CMM are off-base. I think they do show that few developers have any training at all in software engineering practices, but not that the practices are impractical or not useful. There are plenty of studies and data available that show that when organizations do follow the practices quality improves and costs go down. Here at my company I did a study of how much time we spent correcting errors vs. developing new code. Before we put a simple requirements/design/code process in place a few years ago we were spending 70% of our time correcting errors and less than 20% of our time on new features. After putting the procedures in place with reviews of requirements, design, and code we got the rework figure down to below 40% and the new feature figure up to about 50%. The practices work if you want to put them into place.

Recently when I was trying to get my own development team to design their software before coding it, I found that none of them were familiar with the term “functional decomposition”. Nobody knew what a requirements trace was. These are people with computer science and EE degrees but they have no training at all in software requirements engineering or software design. As an industry we are failing to educate our practitioners. Is it any wonder that the SWEBOK and CMM don't reflect industry practices?

I took the CSDP exam last fall and got certified. The point of taking the test, for me anyway, was to demonstrate that I am serious about this business. In most developers I talk to I don't see an attitude that we are a profession at all. A few years ago in conversation with another manager I heard the statement “We don't need any of this process stuff. Our application is simpler than your embedded application, we just code up a few screens, some database accesses. It is all very easy and we can get without any tracking or documentation.” Of course, the product he shipped was buggy and unreliable. I think his attitude is common in the industry and I want to try to change that.

You are correct in stating that there is a hodge podge of practices and standards and there is no one practice that everyone can follow. In a software project management course I took a few years ago it was flatly stated that one must choose and tailor the process to the project because each project is unique. No one process works for all projects. Additionally, the process may require adjustment in the midst of the project. The least we can do as developers is be familiar with the different types of processes people have used successfully in the past and techniques that are available for software quality engineering and project tracking. The SWEBOK is just a catalog of those practices and techniques.

Sorry for being so long-winded. Thanks for continuing to write about software quality and developer certification. We need more voices like yours.

– Steve Hanka, M.S.S.E, CSDP

I agree with you that there isn't a 'single best approach' to software engineering (or almost any kind of engineering) and so, certification seems pointless.

What I do know is that I have attended your workshop (a number of years ago now) and the single best thing I got out of it was code reviews. When the time is taken to walk through your code in front of one to N other engineers, bugs and inconsistencies, where they exist, fall out onto the floor.

So what is the problem? Whether a company uses a formal process like CMMI or a less formal approach like code reviews/walkthroughs the problem is that most don?t actually do them. The paperwork is just a formality and most people sign forms or initial checklists and move on. I think it is ridiculous that a sign would sit out in front of a company stating that they are certified when most of the employees would say that the process is a joke.

I have worked for large companies and small companies. The problem exists almost everywhere (telling you is preaching to the choir but I want to spits this out anyway). I was the boss of my little group at one small company and I couldn't get anyone under or over me to take it seriously. Some people at the top had a lot to say about quality and process but nobody wanted or could belly up to the bar with the money to support it. At another even smaller company I was told that we had to work smart and be smart. We would be a stronger company if we could 'produce' and by this what was meant was get it done without review or fanfare. When a product I worked on turned up with a serious flaw, which was my fault, nobody could understand why. The message was, “You're supposed to be so smart, how could this have happened”? The reason was simple: I'm not perfect. If I had been able to walk through my code; if we had had a checklist, some or ANY kind of review, someone might have seen it. But we had to work smarter and so, we shipped the product with a bug that cost us money. I think it was avoidable. Nobody shared this view. I just kept hearing things like, “This is bad.”

In the previous job I shipped scientific instruments and I never had any reported bug. The company was a little bigger but there, I worked mostly alone. I worked with one guy who was the analog designer and the overall product guy too. I was the hardware designer and the software guy and my work was all done alone and without review. How did I get away without any bugs in those products? They were somewhat smaller in scope than the products I worked on after that which helped. They were a little more involved technically though. I believe the reason they were “bug-less” was that I spent many late hours sweating the details and testing over and over. I lost a lot of sleep, a few years off of my life and when we shipped the product I thought for a few weeks that I was having a nervous breakdown. Was this necessary? In hind-sight I think it was but, how else could this have been accomplished? Could my practices (other than the near nervous breakdown) have been used with a group to get the same results? Could this be turned into a course to certify people? I doubt it.

My experience is that most people don't like their jobs and are just waiting for the weekend. Most people don't take what they are doing seriously and find people that do amusing. The problem is that, we are relying so much now on software seemingly everywhere in oue lives that, like disease and social problems are a natural outcome of increased population, the likelihood that something bad or life threatening will happen with software systems goes up every day. It is inevitable.

Will certifying people decrease the likelihood of catastrophic failures? Maybe enough to raise some statistics but, I believe that the only way this will get better is if we learn and actually apply good practices in our work and if we have employees that care about that work enough to see it through. There will always be some problems and again, statistically, that seems inevitable. But as with our politicians, what we really need are smart diligent people not C students with a certificate in their hands.

– Tommy

Well-done and well-reasoned; Jeanette Nasem Morgan wants software engineers certified like “CPA's and other professions.” The very thought just warms my heart with confidence as I think of Enron, KPMG, and all those other well-certified and convicted professionals! Maybe Sarbanes-Oxley can fix the problem…

– Tim Wolf

The certification idea is a knee-jerk reaction to a perceived problem by those who do not really understand the problem. People areimperfect and will always make mistakes. The really good people make fewer mistakes than the less skilled but that is all you can say.Politicians like certification because it transfers responsibility for problems away from them. All engineering processes should includemeaningful design reviews in which all assumptions are examined, all design is discussed detail by detail, and sufficient time to dothese things is planned into the schedule from the gitgo. I hate to admit it, but most of the design reviews I've gone to hoping for somesignificant feedback, have been busts, usually squandering a lot of time haggling over some minutiae like the font used on the frontpanel. What I try to do now is find one other person who appears to give a hoot and I sit down with that person and do a review. Myexperience has been that the more people involved with a review, the less I get out of it. And that has been more or less the case atevery company I've ever worked for. One company even refused to allow time for any design review (or even product requirement review) andsimply expected that we would make it work somehow. The sad thing is that when it somehow did work on occasion it reinforced the bad ideathat that was the way to do it, when it didn't work, then the engineer was somehow incompetent. This of course led to a great deal ofhard feelings going both ways. I'm glad I don't work there anymore.

– Don McCallum

I have a PE license in Utah, yet find that my practices are not “standardized” in the hardware realm, either. Rules of thumb, goodengineering practice (whatever it is), and following a set of self-imposed neatness guidelines on my work so I can understand it later,are what I have to work with. PC boards, schematics, notebooks, and even my office space need this help.

If there were one way to do it, this would be boring. As it is, I can use one way of coding with a PC/Windows (a little slack, yetreliable) and a slightly different way with PIC, Rabbit or others because of memory space constraints. PC boards need to be made, ICssoldered, etc. Some problems are tool constraints, some are process constraints, and some just good engineering practice, some aredefined by safety or compliance agencies of governments, and some are standards for an industry. And some are just really misunderstoodengineering practice. And some are just real BAD engineering practice. But to make it clear (well I can’t yet).

Maybe we as engineers need to speak up to universities and ourselves about a basic set of mentoring guidelines so we get close to thesame page.

Maybe we need more engineers, and not quick promotions to management without the lessons we need.

Maybe this is why people don’t understand engineers, let alone other engineers understanding us.

– Douglas L Datwyler, PE

Software is engineering design. Bad software is a design problem. Machinery does not solve design problems. It can help find and fixthem, if used. A policy manual in whatever form is just paper. Management is management of people. Management has to have a businessreason for producing good software. Right now, they do not have one, so they will not spend the time and attention to get good software.Certification and design methodologies are irrelevant if the perception is that there is no business case to use them.

Software is in an interesting state in our culture. It has the freedom from responsibility of speech or writing, but has the potentialfor harm of structures and machinery. As far as I am aware, software products in general are immune from being held responsible for theirperformance in the same sense that almost any machine is held to. The performance – or lack of same – of most desktop and businesssoftware would be actionable in a commercial machine.

Perhaps this is a lingering attitude of the early days of software, when it seemed to be an arcane experiment by academics. This isreflected in the computers themselves. Computers are like hobby kits. The customer/user buys pieces of hardware and software, then putsthem together to make them work, sort of. If they don't, it is treated as an ambitious hobby or experiment experiment that ran intoproblems. Too bad, but what can you expect of an academic experiment?

There are cases where this is not true: software in embedded control, such as airplane control software, for example. In this case thesoftware is inside a piece of hardware. If the hardware (airplane) fails because of the internal software, the vendor is in trouble.Boeing treats software the same way it treats airframe design, from what I understand. It *will* be designed to spec and *completely*analyzed before it goes out the door in an airplane.

Maybe that is the the solution. When the buzz wears off, people get tired of useful things not working. They want to buy an appliancethat does what they need without the hassle. As people transition from buying computers and software to buying appliances, the situationcan change. An appliance is something you turn on and watch it work. If it does not work right, you don't care why: you take it back andexchange it. Cell phones and PDAs are examples. These products will have good software inside them. If not, they will be trampled bythose that do.

– Dave Wyland

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.