Write functions using a coding standard. Inspect all new code. Lint your C before testing. Following the precepts of eXtreme Programming, develop tests in parallel with writing the code itself.
All sound advice. Yet worthless for many.
A lot of developers are slaves to an extant code base. They spend most of their time tweaking a few lines here and there to slip in a bit of new functionality or fix a bug. It may be rare to start a new project with a clean slate.
Firmware is the most expensive thing in the universe. A boss managing hundreds of thousands of lines of legacy code will be loathe to chuck the lot and start over unless there's an utterly compelling business need.
It's likely we'll hear: “Yeah, I know it's all crap, but can't you just tune things a bit to keep marketing off my back?” So developers slave away at a code base whose entropic chaos escalates maintenance to a new Olympic sport, wishing they could start over, but bound by the inescapable chains of a ship date.
I suspect this problem will grow worse with time. More developers will become maintenance droids. Fewer new products will start with an entirely new set of code; it'll be more common to torture working software into a different application.
Every year the inventory of legacy code grows; those working for established companies who are both cursed and blessed with lots of working software will be forced to find ways to use the legacy programs as the base for new products.
Like the US automakers who labor under the $1000/vehicle penalty of old pension liabilities, established high-tech companies with big software inventories may start to mirror the rust-belt industries of yesterday. Instead of being saddled with too much antiquated capital equipment, the very technology that established their success will start to become the millstone of failure.
Various authorities claim that firmware doubles every ten months to two years. Where will all that code come from? When a company has millions invested in their legacy products, the pressure to use that code will be irresistible, even when that code might be simply awful.
What do you think? When was the last time you started a project from scratch?
Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at . His website is .
The worst 'offense' is propagating legacy. This is literally “Legacy Gone Wild”! Really, this is the case where one has a really nice brand spanking new platform, new compiler, new processor, just about everything new … only to be burdened with a ton of legacy code. Also, in some cases, this legacy code was on a 'legacy' product that just did not have much in market share or sales.
To tell you the truth, I am very much guilty of generating this 'legacy' code. In ages past, I had a client that first migrated over to an Analog Devices ADSP21xx in the late 1980's. Back then, the 'C' compiler was buggy and not very efficient. Plus, I had this notion that it is better to do everything on a new architecture using assembly language.
F-f-f-fast forward to 2003, and a product discontinuation later. It turns out, my client won several major design wins with the old ADSP21xx architecture, and tried everything in the world to 'kill-off' that product line. During the initial design and feature meetings, I became a leading proponent of 'junking' my old legacy code. However, management, and even the embedded dude who was going to take over told me that they were very happy with the 'scanning engine' of the code and wanted to save that. So, later, I reviewed all that stuff that was done in the late 1980 … early 1990's and came to the conclusion that the 'scanning engine' could be encapsulated. So, this is what eventually happened:
– New hardware using the latest generation of Analog Devices ADSP21xx designed in
– Added new hardware to enable JTAG support
– Added new hardware to give visibility to all processor subsystems
– Added new hardware to give some embedded diagnostics and manufacturing diagnostics
– Created a real-time 'C' shell on new DSP
– Debugged and brought up the new 'C' shell
– Brought on-line a DSP Command Line Interface
– Figured out how to encapsulate the old assembly 'scanning engine'
– Ported old assembly 'scanning engine' as a real time pre-emptive service
Fortunately, the 'scanning engine' to this day, has never needed any refactoring or code modifications … (cross my fingers!) . That was now almost three(3) years ago. I say kudos to those that can effectively encapsulate the stuff that works, and for those that have the courage to toss out entire sections of code.
Ya gotta have the courage and fortitude to do either one!
– Ken Wada
Just about every project I've completed (using the term “completed” loosely) in the last six or seven years has involved beating a big, steaming pile of spaghetti code into submission (chock full of errors such as dereferencing NULL pointer or accessing data past the end malloc'd blocks — stuff that didn't cause DOS to crash).
It's the pits. Seeing this article makes this fact even more depressing.
Hmmm…. between this, committments to customers for features that may not even be possible (especially given the legacy codebase and hardware), cubicles, and all the rest… it's amazing that so many of us stick with it (I suspect the stack of bills that shows up every month has a lot to do with it).
– Robert T
We've all endured the pain of incorporating a proverbial “big ball of mud” into a new product… but it's important to remember that code was “new” at one time. So, why did it become the millstone it is today? More often than not, it's lousy design. We have so many tools, in the way of sophisticated languages, configuration management software, etc. at our disposal, that there should be no excuse for developing rigid code that requires punishment to adapt to a new application.
(Nearly) every new project entails *some* amount of newly-developed code, whether it be embedded software, a new HDL module for the FPGA, etc. When a cohesive new piece of code is written (e.g. a device driver, algorithm, hardware peripheral or acceleration unit, etc.), treat it like a piece of intellectual property. Give it its own development area in your source control repository, and design it with flexibility in mind. As you develop your product, include the developing module into its CM database *by reference*, not by copying it over and setting yourself up for the fun of maintaining a codebase that's fragmented all over the place, with different bits of Krazy Glue and chicken wire slapped onto it to make it stick to applications X, Y, and Z.
I'm not advocating designing in all sorts of extra functionality for the future; the XP'ers are correct on resisting that particular urge. However, if there's an abstraction that makes sense and decouples the code, implement it. If a state machine or data type in your HDL can be made generic, do it. A piece of code only tends to become truly reusable after it's been reused about three times. As the IP matures, make meaningful release tags, and point each project which incorporates it at a specific, tagged release to isolate them from the changing trunk.
– Eldridge Mount
Fred Brooks would have loved it: The five thousand, four hundred and thirty Second System Effect.
This discussion just warms my heart, being involved as I am in the business of managing a very dynamic inventory of over thirty thousand increasingly software-based medical devices used in a couple thousand bed system of academic and community medical centers.
To those engineering managers who choose to take of their engineering hats and put on their mangement hats, and particularly the one who is bound to end up holding the hot potato: See you in court.
– Rick Schrenker
That's life for many embedded automotive software projects. First started as a toy or research project, then sales put it in front of customers and given some tweaking (and no formal requirements) it got accepted as is.
Then our company got sold, the engineering outsourced to a country not familiar with embedded automotive pand the customer wanted to adapt the existing product to a new vehicle platform. Done deal with sales, but on the engineering side sheer mayhem since no knowledge about initial requirements was conserved, “C” code without comments let alone proper documentation needed to be “reused”, while time and budget was according to the very first tweaking. Although bringer of bad news, I survived that project somehow.
Latest I heard about that product, for the next vehicle generation the customer gets completely new product, new processor, new development process. And happy engineering.
– Bernhard Kockoth
Too well have I heard these words “I know its been ported twice but we have so much there, it needs to be ported a third time…”. No documentation (the original coders were too busy or lazy). Might as well become Italian with so much spaghetti to wade through. Written in C? HA!!. Poor code structure is still poor code structure, no matter what language.
You cannot make chicken salad out of chicken droppings.
Give me a project with a clean slate so we can properly document and develop it.
– Keith Mullins
I keep a rather pragmatic outlook toward legacy code. After all, revenue from these products typically keeps the doors open and the lights on; it also affords us the luxury of developing new toys.
Given the opportunity, I thoroughly enjoy developing new products. These days however, my team primarily maintains legacy code. Fortunately, this code is relatively clean, well-designed, and intelligently commented. Of course, we still find the odd skeleton tucked away in a dark closet. When compelled to touch these “aesthetically challenged” sections of code, we endeavor to improve them. Often, it's simply a matter of changing a few variable names or cleaning up the comments. Occasionally, we must re-factor an entire module.
Before committing to a release schedule, we always take this non-value added work into account and budget accordingly. If management is reluctant to accept this necessity, we do it anyway and don't tell them; the schedule gets padded accordingly. If we defer dealing with the skeletons, they always seems to come back to haunt us later to stretch a metaphor.
I've been supporting and augmenting our legacy code for too many years to admit, but I take pride in the fact that each new release is better than the last. We rarely have field issues; when we do, the bugs are usually found in code we haven't yet touched.
If you primarily develop code for new products these days, you'd better do a good job. Like many of us, you may be maintaining it for years to come.
– Richard Rogers
Your argument runs counter-current to the widely held argument that we don't reuse code enough. If we aren't re-using code, we must be creating new code. I work for an avionics company, excepting reuse of a CRC calculation we don't carry over any code from one project to the next. Each project is too unique so 98% of code is application specific. I once worked on a legacy avionics platform that was targeted for a microprocessor upgrade. I was able to totally greatly refactor the code to a object oriented architecture. Then the airframe manufacturer decided that upgrading the hardware/software would be too difficult/expensive to FAA certify. So my code sits in CM dying to get out. Pretty funny.
– Phil Gillaspy
Working on a code base written over 15 years, mostly by people not with the company any more, maintaining all that is bringing forward progress grinding to a halt. The release cycle for new versions with new features is getting longer and longer.
I think the right way is to start with a blank slate, building up a new architecture for the problem as we understand it today, but bring in components from the old code base as appropriate. The old file format parser is just fine, but the event model needs to start over.
– Noonian Soong
Beauty is in the eye of the beholder. Every individiual and team has its own unique style of coding. Anyone introduced to that code will find it ugly in some fashion or another – unless they can take on the culture and context of the original authors.
In other words, I agree that we'll spend most of our time working on legacy code, and that we'll invariably find it ugly.
What we really need is to capture the intent of the code – the reasoning behind it – in the unit tests. Subsequent generations of designers will have a better appreciation of what was done. They'll have the courage to refactor whenever necessary because they've got the tests to prove that their new code behaves exactly as did the old code.
Does this apply only as we write new code? Absolutely not! Any adjustments to legacy code should be preceeded by the creation of a firewall of unit tests around the effected code, articulating and asserting its behaviour. The subsequent refactoring is then less prone to error, and ends with a fully tested section of code.
– Dana Smith