James Bond never had a plan, but through the miracle of fiction always managed to save the world. Never bumbling, yet never with a clear strategy, he somehow forged straight ahead to get the girl and put a smile on M's face.
Contrast that with building firmware. In the 50+ years of computer history we've learned to build code by first going through an excruciating design phase, and only afterwards creating piles of C. All of software engineering is predicated on the belief that code is very expensive, and change even more so. Craft a perfect design up-front to minimize modifications later.
Yet, if we're really honest, we do know that design doesn't quite work. How many projects end up with software implemented per the original design? Even with unchanging requirements, we're mostly unable to anticipate all of the problems that arise during development.
Worse, how often does the spec change during development? Specs are inherently fluid, changing as the marketing weenies constantly make revisions since, after all, “it's only software”. A mid-project spec-change means a late scramble to adapt the code – not the design – to the new requirements.
Those who espouse eXtreme Programming (XP), a rather new approach to building software systems, see design as a source of problems and suggest lowering its importance. The four activities in an XP environment are coding, testing, listening, and designing, in that order. One mantra sometimes heard from XP zealots is “the design will emerge from the code”.
XP promotes the idea that we can write a lot of code very fast if we're not restrained by the normal front-end efforts. Crank it fast, but be willing to change it constantly. As issues become clearer, “refactor” (rewrite) constantly.
The ideas are fascinating, but the old fart in me trembles at the thought of forgoing or de-emphasizing design. I just can't imagine how structure can evolve from the chaos of the bits and pieces. Yet, deep in my heart, I can't think of any system I've worked on where changing requirements didn't mean a lot of late code changes, rarely annotated back into the design or design documents. All too often we crammed in major structural revisions very late in the project.
Perhaps XP is a tool best applied to smaller projects, those devoid of complicated interactions or communications with many other processors, processes, or systems.
Or perhaps it's best in an environment where the result is crystal clear, where the team members have built the same kind of system before and need less guidance from formal designs. Maybe if you build colorimeters all day long, and know everything there is to know about colorimeters, the design phase may inhibit rather than enable progress.
There's a lot more to XP than I've space to discuss. Programmers work in two person teams, intimately sharing a single workstation. Test code is as important as the product, and is written concurrently with every bit of software; as the code changes so do the tests. That's very cool.
XP projects always have a customer who lives with the development team. Not an occasional visitor auditing progress, but as an essential team member. The customer is the safety valve, insuring that the system we're building will indeed solve his problem. Adherents feel this compensates for the de-emphasized design process, which may make some sense.
But without a design, how do we anticipate hardware requirements? Is scheduling possible?
In talking to hundreds of embedded companies, I've come across only one that claims to practice XP, and in that case they employed only a subset of the process. Is XP viable for embedded designs?
What do you think? Is abdicating our up-front design suicide, or a healthy recognition of realistic practices?
Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. He founded two companies specializing in embedded systems. Contact him at His website is .
Prototyping is part of any good design process. Prototyping allows the little nagging problems such as compatibility between libraries to be discovered early. The XP design process is nothing more than a prototyping heavy design process, which, I believe is good. When designing a mechanical device there are real physical expenses that are incurred when prototyping, but, for software, the only expense is the time spent. People often refer to the man-hours of coding that can be saved by a good design up front and this can be true, but what about the man-hours of designing that can be saved by some good coding up front?
Jack Responds: Agreed… but an important characteristic of a prototype is that it is thrown away…
First and Last, cool design is an art. There are various points in a project cycle when failing to slow down to do formal “big” design means your product will fail — unless you happen to be lucky — or be only average. These are the well-known design “points of no return”. There are also points when everyone pretty much knows how the design should proceed, and should be programming in a XP fashion. Because I help maintain/extend an embedded telecom code base of over 500,000 lines of source, the problems most apparent to me are design problems. Whatever methodoligies we programmers may expound, the bottom line for medium and large projecs, IMO, is that you MUST get the macro design (interfaces::now and future, features::now and future) pretty much correct at the beginning, or at least learn enough about Object-Oriented Design to provide for product expansion. Not doing this means spaghetti swamps of code when produced in the last 1/3 of the project cycle when everyone !starts to aim their code towards the way it should have been done in the first place.
I agree with the XP “test often” mantra. I never code for more than a few days, maybe week at most without testing the whole shebang.
But my Bottom Line on the importance of good design:There is no way to undo cooking an egg, so decide at the beginning whether you want fried or scrambled.
Embedded S/W Engr.
XP sounds too much like the Reality TV that we see today. Skimping on design looks good on paper but it may cost you later on.
Simply put: How can you successfully code when you don't have a strong design to code to?
Answer: You're coding something you've done many times before.
If you have a big project, then doesn't it make sense to think about as many things as you can up front before you devote many man-hours to coding. Even in small designs you will miss things, but the fact that you spent time creating a design means that you will more easily be able to take care of those cases or things that were missed in the design.
Design and Coding can be an interative process. Create the design and then implement it. During the implementation your insight will grow and you will be able to expand the design. You can then go back to upgrade the design and keep coding afterwards. This works very well for me. It's an interative process, but I need that initial well thought out design otherwise I'm just foolishly pounding on a keyboard. The thought of those gray areas that I didn't work out would make me nervous as I code.
In short, I'd rather sweat up front and cruise later.
Management though may have different thoughts. Getting the coding stage running will make a gant chart look better.It seems like progress whereas sitting around and thinking about a design or having design reviews doesn't produce any results immediately – it may cause people to “go-back” and re-write or improve their designs. In my experience, this is not what manager wants.
I don't know who made this observation but it it true of any software development.Also, there's no cheating it.You can only pick two items: cheap, fast, and working.
Senior Software Engineer
As Jack mentioned several times, XP does not eliminate design, only de-emphasises its importance. Too often, big-design up-front (BDUF) approaches like the waterfall model impede progress, and discourage change later in the cycle. XP embraces change, but more importantly emphasizes the need to begin somewhere immediately.
Some important tenets of XP:
Get something running, no matter how small. Elicit requirements from your customer, and (real important) have him/her *prioritize* them. This may be new for them; all their requirements are important, but which add the most business value? Begin implementing one use case at a time, designing and coding only the framework that is *immediately* necessary. Work in small, fast iterations to evolve the software, and revisit your design and assumptions often.
Don't design for tomorrow; rely on your ability to adapt, and refactor your code mercilessly. If the code can be simplified, stop and do it. Avoid code duplication like the plague; once and only once. If all our efforts at creating better OO languages and tools have paid off, we should be able to experience a flattened curve of effort to change the code vs. time. OO is a major enabling technology for XP.
Write tests before you code the solution; this will *increase* your velocity and bolster your confidence in your ability to refactor. Integrate often, whenever all the tests pass. Work in pairs; this improves the thought process and reduces the tendency toward “code ownership”.
XP was never intended for use in huge projects, but works very well for small to medium projects where constant communication is possible.
I think XP would be ideal for maintenance mode in a complex system, mainly because of the emphasis on refactoring. The concept of a system becoming less complex and easier to understand (evolving) during maintenance is opposite to what I have observed using typical methods, especially when staff turnover occurs.The “de-emphasis” on design makes me uneasy to consider using it on new (from the ground up) projects.
It is not true that the XP processdoes not have design. What XPdoes not have is a BUFD (big upfront design). You do not do adesign for more than can be codedin the next cycle. Cycles are therange of one to three weeks tops.You will lay out a design for thefeatures to be implemented in thecurrent cycle. Presumably yourrequirements will be stable forat least that length of time.
XP also has “spike solutions” thathandle figuring out the bestsolution to a problem that youhave not solved before. These aredeeper solutions that give youthe confidence to move forwardwithout the BUFD.
I do not believe it is correct tosay the “design will emerge fromthe code” there is some thoughtgiven before sitting down to type”public static void main”. Whatis true is that the design isdocumented in the code.
Rochester Institute of Technology, Department of Software Engineering
For all of the people who advocate extreme programming, how many are willing to get aboard an airplane with software that was developed in this fashion. While it is true that a software design will rarely anticipate every need or problem, it seems to me that just piling code up “until it works” is not a viable solution.
Another thing that the design gives is a structure to the project. Who works on what features? When is the project complete? How do you test the features in the software? Without a design, it seems to me that these questions cannot be answered.
In my past experience, the projects which have taken longer than they should have, had more bugs, had more bugs discovered by the customer and generally been crummy ( a technical turn used by old guys like me) are the ones where we jumoed into coding (don't worry, I've done this same type of project before) with little or no design.
In the early '80's I worked at an company in the military communications industry. The hardware was a main cpu with about 20 8080 subprocessor's connected to it. The comapny had a dedicated staff of about 20 people for collating and disseminating documentation on the design – of the hardware. You could walk in and get anything you wanted – how the cpu's communicated, schematics, anything. This was a complicated system – rack mounted cabinets which filled a 2000 sq ft. room. Since it was a military project, everything physical was documented – you couldn't find a scrap of paper in the building that described how the software/firmware was written. We had 500,000 lines of code, 200 software engineers, and no clue of where we were going.
Scary, isn't it….
Director, Applications Software Development
Global Payment Technologies, Inc
Jack Responds: Caper Jones, in a study of 4000 s/w projects, found that poor quality was the #1 reason for schedule slippages….
I've worked in embedded systems –initially in hardware design– for almost 20 years and, like Jack, I am quite uncomfortable with the current trend towards attemting to produce software without a preliminary design phase. I am largely a 'self-taught' engineer, but I was fortunate to have apprenticed with a couple of engineers early in my career that were very methodical in there approach to the design process. Even when I'm working on a relatively simple product or feature I will typically write a one or two page document describing the basic design before proceeding with any coding. I can't help but believe that this clarifies my thoughts regarding the design and results in a shorted design cycle. I do, however, believe that some companies require design documents that are overly complex and lengthy. This may be the area that would benefit the most from XP techniquest.
As u said that the four activities in an XP environment are coding, testing, listening, and designing, in that order.I disagree with this order.
The order can be listening,coding,testing,designing.
What I personally believe is that before we start coding we should understand the problem domain well.i.e,Listening
I further didn't understandaccodring to XP design was done after coding & testing i.e, after your system is working fine.
Why do we need design at this stage?i.e Getting design after implementing a working system.Is this for maintaining the system at the later phase?
Balaji Sunil Kumar
Jack Responds: Very good point about listening! Truly critical.
But XP is not for after the system is “done”. It is supposed to be applied from the git go.
There are many software development paradigms, and each has strengths and weaknesses.
The classic waterfall model is excellent when all requirements can be specified up front. It is death when you need to spin a cycle or two just to define the problem!
XP is similar to a spiral development model, except more extreme. XP would be death if applied to situations where the waterfall model is appropriate.
You wouldn't buy clothes at a store whose attitude was one size fits all, would you? Why should your development model be one model for all projects?
The trouble with the fad-ity of these things is not that XP has problems, but that it will be applied where it shouldn't be. (Just as waterfall is often applied where it should not be!)
When a screwdriver breaks while being used as a crowbar, that does not mean it was a bad screwdriver.
Kevin G. Rhoads
Jack Responds: Absolutely! I also think a wise developer will learn many methods, and pick and choose the practices from each that work best for him.
This technique reminds me of the algorithm of evolution. Stable population (base code), modification (new revisions) with selection (user likes it).As long as you are building on code that tested “good” you can incrementally, gradually, get better. It does take a lot of pruning or testing. There is the checking or testing of the programmer as they write it closely coupled with the final user's testing. Is the code a “good trick”?It also takes lots of generations or revisions. If the programmer doesn't even know what is needed his checking becomes important in that the code does something predictable that is easy to change. This allows for lots of generations to be pruned by the user.Does anyone else see any other parallels?
Jack Responds: Good question; any other parallels?
The really important thing is to have different levels of abstractionand a way to go from one level to the other without loosing anythingIf vou have a gap between design and code, coding will loose theconnection to the design. So it makes sense to close the gapby designing directly in code.But don't loose your levels of abstraction!
Embedded Software Engineer
It is a judgement call. Some projects may not need much of a design phase and code may be hacked out and the project completed quickly.
But usually there is no substitute to understanding the problem and its solution up-front first and then coding. That should be the goal. However, it may not be possible practically. That is where some code has to be first written to get a handle of the situation and then go back to design. I believe the whole process of design-code-redesign-recode is how it should be.
I don't know any project where the design was so robust that it needed no change at all. However, I've also seen myself hack away at code for days without a design and end up with very bloated ineffecient code which could have been made simpler and more effecient had I understood the problem and solution better.
Therefore, I believe that based on the problem at hand the engineer should decide how the problem has to be tackled. If the problem is fully understood, then start code, else think-design-code-test-rethink-redesign-recode-retest until project is complete. No hard and fast rule on how a project should be completed.
Tucson Embedded Systems
I read the article “An Introduction to Extreme Programming” and the replies posted, and found that most felt as I do, XP is not new to experienced programmers or Software Developers. Except for the pair programming it is very similar to Rapid Application Development (RAD) or Spiral Development. The pair programming idea is novel to me, but I think regular code reviews are likely as valuable.
I disagree that they are advocating elimination of all design, it seems XP advocates a reasonable approach that is more in line with the way most teams work.I'm all in favor of anything that advocates better coding practices, and applied with the proverbial grain of salt, I think XP can be beneficial.
Jack Responds: Oh, I said XP deemphasizes design, not eliminates it. That makes me quesy. But… there's a lot about XP that I find quite fascinating.
You may like to know what XP looks like to one standing on the ground looking up at the mountain.
1) eXtreme Programming is akin to Extreme Sports.To be taken on when one excels at those mundane things like planning and design. Been there, done that. Who wants to be roped on the practice wall when they can free-climb up mountain without a rope?
2) They do design, but as a sculptor can design with the clay, they design with the code, and *for them* coding, testing and refactoring makes for better and faster work than translating from one abstraction (say UML)to another.
Presumably the quality of their software bears them out.
I can't really see any other way XP could work; no design is chaos.
Drawing is one of my passions, and when you really hit the Zone, designing and executing become the same thing.
Could software writing be done like that?
Well I can dream, can't I?