Code guardians - Embedded.com

Code guardians

The nuns of St. Camillus pounded all sorts of interesting ideas into the heads us grade schoolers. Some will no doubt come out in the inevitable therapy sessions I’ll need to get my kids through teenagerhood. Others, though scary at the time, make for good stories today.

“Guardian Angels” were the nuns’ invisible avatars, always watching over us, often helping us, sometimes recording black marks in that cosmic book of personal errors. No doubt my guardian angel is a UNIX guru, working full time just to keep the sheer volume of my black marks from crashing all of heaven’s database engines. “Reboot heaven? Y/N?”

The nuns neglected to tell us we’re each others guardian angels. Parents watch over their kids with a wary eye no supernatural creature can match. Spouses look out for each other’s interests. Friends in need are friends indeed.

These thoughts bubbled to the surface when I asked an engineer to recode a few functions in assembly language. The system worked great but just seemed a bit slow. Some analysis uncovered a busy interrupt service routine, supported by a couple of equally stressed main-line subroutines.

I wasn’t looking for much speed–a 20 or 30% improvement was more than enough to give the system a snappy feel. Why not translate a bit of code to assembly and be done with the problem?

The engineer’s boss collared me, and in no uncertain terms said “No!” Though he knew that the translation would work, he figured the resulting assembly language–the only non-C code in the system–would reduce the system’s maintainability.

He was the system’s code guardian. As such he had the authority–and the common sense–to ensure we don’t do dumb things that erode any aspect of the software, software we’ve invested in a big way. Though I might disagree with his approach, he watched over his code like a parent over children. He protects it from the whims of marketing types, and from expedient solutions like mine.

The code guardian is the quality angel. When a dozen approaches will all work, the code guardian selects one that most closely meets the needs of the customer, yet that preserves the software’s integrity.

Every project needs a code guardian. Usually this should be the team’s highest-ranking technical person, someone with the authority to say “This stinks! We’re not going to do it this way!”

The code guardian needs the freedom to do work management might consider non-productive, like instituting version control systems, developing and maintaining software standards, and sometimes rejecting working routines that just don’t measure up to his or her demanding standards.

It’s hard to chuck working code. But sometimes it’s simply the right thing to do. The Code Guardian has the guts–and support from management–to make these hard decisions.

It never ceases to amaze me that companies hire huge accounting staffs to guard financial performance, and security firms to protect physical assets, yet seem to neglect the critical importance of their software. Since you can’t see or feel code, management leaves it on its own to wither or thrive, protected only by the techies who understand it.

The role of the code guardian is to protect the software from outsiders (marketing) as well as from the developers themselves. Most projects start with the best of intentions: code reviews, structure charts, careful design are our mantra when delivery is a year away.

After a few schedule slips, with the deadline looming, it’s all too easy to abandon a reasonable methodology in our panic to get a product out the door. The code guardian institutes a discipline of methodology. He or she becomes the code Gestapo as the crunch hits, insuring that things are done right. Bad code always comes back to haunt us.

One tenet of eXtreme Programming is “code that can be improved, must be improved.” XPer’s refactor mercilessly. In a sense everyone becomes the Code Guardian. Though I worry that unrelenting refactoring fulfills that much-reviled (by managers, at least) engineering need to fiddle with a product forever, I admire the XP idea of keeping a clean code base.

What do you think? Is the integrity of the code important enough that someone has the authority to reject expedient but ugly shortcuts?

Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at . His website is .

Reader Response


Absolutely!! I have seen too many bugs that cost the company weeks or months of effort to debug. Often the root cause is so obscure that the person or group that introduced the bug doesn't even take part in finding the issue. The rush to ship is most often the cause . . .

– Kurt Lumbatis
Sr. Software Engineer
Cisco Systems, Inc.


In the situation you described, how 'bout fiddling with the C source until the compiler produced faster code? Many times I've been able to speed up a C routine by examining the list file, adjusting the source, and back and forth until the compiler gave me what I wanted–things like changing array references to pointers, or vice versa, depending on the compiler. No need to get anyone's approval or fight with the asm/C interface.

And yes, sometimes that meant more cryptic code, which I tried to mitigate with tons of comments, including the non-cryptic C lines.

I'd be interested in your and fellow readers' reaction.

– David M. Tomer
Senior Software Engineer
Bosch Security Systems


One reservation I have about the mandatory refactoring mantra is that every cowboy coder who comes along always thinks he can do it better. In fact, all of the descriptions I've read about Extreme Programming fits what I call Cowboy Coding–shoot first, think later.

– Larry Dingle
Sr. Staff Electrical Engineer
Thales Avionics


In your example, simply inspecting the assembly language output from the compiler (followed by some experimentation with the high-level source) may well have produced the desired optimization. You then have the best of both worlds–fast and maintainable code. To answer your final question, I do think code integrity is generally more important than expedient shortcuts–but let a peer reviewer decide.

– Phil McKerracher


I think there's no need of a Code Angel, instead of that it will be interesting a discussion in the software area about the needs of this new piece of odd code 😛

A well documented code can be written in any language (maybe here is the difficulty: documentation)

Best regards!
Manu


Many times I have been yelled at, cajoled, and technically thrashed to 'preserve-at-all-costs' some code or hardware that I originally think is most dubious. Only to find later that preserving such ilk has caused massive hemorrhages later on in the development and deployment . . .

only to be accused of causing the problem in the 1st place!

and . . .
Much later; After showing that the problem was originally with an extremely poorly implemented or poorly architected piece of #*!@!!!

Then, I challenge the managers as to why such things were allowed to promulgate . . . I get the inevitable and much predictable answer:

“Well, it always worked in the past, and we have always done it that way!”

(this by-the-way is a classic case of 'brittle thinking')

only to be followed up by:

“You know Ken, you bring in so much complicated technology to bear, it is no wonder that you keep breaking things!”

sheeesh!

– Ken Wada
Sr. Embedded Systems Consultant
Aurium Technologies Inc.


The “Guardian” you speak of must be an Architect, who protects the whole project, from the beginning. Just watching the code isn't enough. When I've worked on systems with a strong architect, I've seen great results appear.

Other times, “when the (architect's) away, the mice, will” play? No, rather, run amuck, with hardware changes and software requirements popping up at the end of a project when it's about to ship . . .

Once that level of disarray has occurred, a Guardian Angel can only lead the project to it's final resting place.

Competent oversite is good, but it must start early and have broad application.

p.s. I really mean competent oversite, not some bureaucratic scheme to handcuff talent.

– Randy Leberknight


Jack, didn't you have an article very similar to this one about 10 years ago? I don't remember the XP references :), but I distinctly remember reading this article. Or maybe the pressure has finally gotten to me.

– Tad Ashlock


Tad's right–I remember this from before. Maybe while Jack is off cruising the Caribbean his Article Angel inserted this.

Or maybe Jack really believes this and it's a lesson we need to hear again.

Either way, I'm with the “figure out how to make the C better” group, resorting to assembly only as a last resort.

– Andy Kunz
Sr. Firmware Engineer
Transistor Devices


In addition to filtering tweaks to the code, the guardian is also the one that decides when it is time for a shake-up (my motto–“It's never too late for a major redesign”).

– Dave Alsup


Hey Tad, don't visit your shrink quite yet:www.embedded.com/97/toc9607.htm www.ganssle.com/articles/codegd.htm

– Fuzz Memory


Interesting that your same comments apply equally well to documentation. I am the data book guardian. Before I arrived, engineers wrote sections of our microprocessor data book in whatever style and whatever jargon they liked. After I became the guardian (editor), their prose had to pass muster or be rewritten for clarity, accuracy, and brevity. It's amazing how many people can verbally describe something complex like a processor MMU but they wrap themselves around the axle when they try to write up the same ideas. Their university training taught them that writing the way you talk is “unprofessional.”

All significant aspects of a project need a guardian, as you pointed out so well in your column.

– Steve Leibson
Technology Evangelist
Tensilica


Does XP really permit each developer to redesign his/her portion of the system at will? I would hope your “code guardian” would control not just the code but also the design, requirements, documentation, build scripts, etc. Otherwise, you're not developing software; you're codifying chaos.

P.S. My “requirements creep” alarm went off when I read your anecdote. Did the spec say to make the response time “snappy”?

– Gary Heiligman
Senior Professional Staff
Johns Hopkins University Applied Physics Laboratory


It all depends on having a solid software development process. Toss all this XP crap out. Follow a well-documented process and things will be fine. Avoid code that cannot be maintained at all costs. That would automatically get rid of the code-nazi. I believe code-nazi's are more of a hassle than anything else. If you still feel you need a code-nazi get an SQA instead.

– Sid Ray

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.