It's too bad Andy Rooney isn't a software engineer. He could really have a field day with all the people whining about the sorry state of software development. Software is too expensive, it takes too long to create, it's riddled with errors, and it's impossible to predict when it will actually be ready to ship. As a result, we're up to our eyeballs in software-development platforms, frameworks, methodologies, best practices, management tools, training, documentation guidelines, ISO, CMM, consultants, outsourcing, and more. I've never worked for a company where some senior manager didn't say “I hate software.”
So let's take a step back and look at the present reality of the world around us. Businesses trade, Game Boys entertain, jets fly, and we communicate with one another, all with the help of software. The stuff is everywhere and it works. Is it expensive to develop and deploy? Yes, but so is a chain of department stores. Is it difficult to schedule? Yes, but so is a highway construction project. It really doesn't matter what a people are creating. Whether it's dinner for 200 diplomats or a new pulse oximeter, there are going to be surprises. Perhaps it's your introduction to a Texas Instruments digital signal processor and you didn't know what happens when you misalign a floating-point number. Or you're involved in a software conversion to .NET and you didn't realize that header files don't use the .h extensions anymore.
It's impossible to plan for the things you don't know about–situations that invariably cause development activities to take longer than you planned. These delays generate angst among managers, and that's natural. But methodologists don't cure angst. You fight angst with therapy. So before you go off looking for tools, people, and practices to solve your problem, consider a therapist. They're better at it and they cost less.
Okay. Say you've tried therapy and are now committed to improving the development process instead of agonizing over it. First–and I want to be very clear on this–no method can help if your organization doesn't actually adopt it. As self-evident as that may sound, companies spend many thousands of dollars on method and process consultants, only to ignore their advice. I've seen commitment to methodologies fall apart when it came time to actually use them. I've also seen managers bring in trainers to introduce the Capability Maturity Model (CMM) to their organization only to vanish once the training is over, leaving the engineers with a program that devolves into a twice-monthly “best practices” argument over a catered lunch. And why would a group of intelligent, motivated, and thoughtful software engineers not take a few days training and inscribe it in their hearts, minds, and cubicles? Because methodology is a management issue. It's the manager, above all, who must understand the process.
Here are some symptoms that dysfunctional development teams often manifest:
- Low productivity
- Poor communication
I've seen proven methods that address each one of these symptoms. If a manager has a particular symptom to focus on, he or she can choose the method that best deals with the problem.
Take chaos, for instance. Chaos, as we know, happens when the project team has no process at all. One recent example I saw occurred when a client needed to collect and store 10KB of data per second per channel, for each of eight A/D channels, in real time, while performing complex math. Unfortunately, the processor was running at just 40MHz and had only a single serial port for data handling. Simple arithmetic shows this is not a great situation. A simple review of requirements (had there been such a thing) and past projects might have led to the incorporation of a hard disk drive in the system prototype. In fact, the lack of clear requirements was reflected in the overall development effort, with managers hoping for project completion without understanding what was necessary to bring this about.
For this type of situation, I suggest taking a look at the CMM from the Software Engineering Institute at Carnegie-Mellon University. So much has been written about this method that I won't summarize it here. However, if you're going to introduce CMM to your people, be prepared for the extra work it entails.
To wit, if you're at CMM level 2, you have a defined process for developing software: a management system. This sounds great until everyone realizes that, to allow people to follow your process, you need to provide documents describing that process. Should you desire to subject yourself to audits, you'll need documents to prove that you're following the process. If you don't manage the documents properly, an inordinate amount of time gets spent writing and reviewing documents, which becomes an activity for its own sake.
Balancing the need to get software done and the need to prove that it's being done according to process is a new concept for many software-development project managers. The payoff, however, is that for the first time you might understand exactly how much work has been done, how much is left to do, and about how long it will take.
Extreme Programming (XP) is an excellent method for increasing programmer productivity. According to Kent Beck, XP requires a few very specific practices:1
- Programmers work in pairs.
- A customer for the software is always available (even onsite) to the development team.
- The developers all work in the same room.
- Food is always available in the room.
- Programmers are always working on a new bit a functionality required by the customer.
- Test code is written before the functional code.
- If the existing design doesn't support easy incorporation of the new functionality, the code is refactored until it does.
- Test code is continually run to ensure that refactoring has not broken anything.
Each one of these practices exists for a very specific reason in the quest to improve productivity. First, by isolating programmers while inviting interruption, cubicles can kill productivity.2 Programming is a community activity. It requires cooperation to create a successful project. Food nearby prevents disruption, encourages casual discussion, and generally makes engineers feel good. When programmers work in pairs, it's less likely that they'll waste time on the Internet, planning lunch, or programming down a rat hole. A full, automated test suite guarantees that the software works. Having a customer (I prefer “product champion”) accessible to developers is extremely valuable in the absence of clearly written, stable requirements. Rather than have such a person write specs, having the customer around lets programmers resolve important requirements questions as they arise.
XP's main challenge is that, unlike CMM, it requires a leap of faith to adopt. A common thought process generally goes along these lines: “If my programmers aren't working fast enough, how can I have two programmers working on the same code? That will slow development by 50%.” The act of writing software using the XP method is also not intuitively obvious and benefits from training and consulting. Coming up to speed with incremental development, which involves writing tests before writing code, doesn't happen overnight. It requires practice and patience, which means the manager must create a supportive environment that allows programmers to gain experience.
Finally, software engineers have a notoriously difficult time communicating design concepts and development progress to their managers. Fortunately, the Unified Modeling Language (UML) goes a long way to resolving this problem, were people to use it. Going a step further, Model-Driven Architecture (MDA) takes advantage of the UML and a variety of important concepts such as domain analysis, platform-independent architecture, and object-orientation, to create a software engineering practice that rivals electrical and mechanical engineering. model-driven architecture.
MDA suffers from the fact that very few people have ever heard of it. While the Object Management Group is beginning to make a bigger fuss about it, MDA is still a development methodology on the fringe. That's too bad, since, by supporting design representation and code reuse, it's great for companies that develop a line of products with similar functions.
Using models to describe a system is nothing new in engineering. Automobiles, bridges, and golf courses are all designed using models to give the stakeholders an idea of how the finished product will look. Models can be built, reviewed, and refined at a fraction of the cost of coding. The good news about modeling, according to MDA, is that these models, once accepted, can be used to generate application code. Not only does this work, but it's in current use on medical, military, and industrial automation projects.
The steps taken to develop an MDA project are fairly logical, but it requires a fair amount of study and practice to see that logic. The first step is to partition the system into domains of subject matter. If you employ motor controllers, that's a different domain than user interface or I-beam transport. Separating responsibility of software modules by domain ensures that the system is minimally coupled; changes in requirements to one domain are not likely to affect another. After domain discovery is complete, you story-board scenarios that describe how the domains interact with each other to accomplish important system tasks. Once these system-level scenarios are complete, the next step is to do an object-oriented analysis of each domain. Certain classes within the domain are given lifecycles that are described in state charts. Finally, each state in a state chart has an action. It's here that the actual implementation details get coded.
Aside from the obvious benefit of having a clearly designed system, development teams can readily reuse code at the domain level. This means that a company that makes electromechanical devices can develop strong electromechanical support systems that can be integrated through the model editor with relative ease. This design strategy also makes it clear into which domain application-specific functionality is to be placed. And of course, the diagrams are an effective talking point for discussing what work is done and what remains to be done.
If this method is so great, you may ask, why has no one heard of it? My guess is that software system developers don't routinely think of themselves as architects, and this approach is, as the name implies, an architectural approach. Much of the work on the system has nothing to do with the application per se . The architecture consists of a state machine engine capable of coordinating multiple simultaneous state machines through an event dispatcher. This is a departure from the normal application-specific thread-based development schemes usually associated with Windows or RTOS-based designs. Like XP, MDA requires practice and patience while facility with development and debugging is achieved. MDA requires the adoption of domain separation and object orientation as two key concepts. From an engineering support (what I like to call scaffolding) perspective, the organization must now manage models in addition to code, and most developers who are quite comfortable using a text editor become squeamish or downright rebellious when they must use drawing tools for the precise work of system definition. (The phrase I find apt is “abstract but precise.”) These drawing tools are improving, but ultimately, working on one drawing at a time isn't like working on one source module at a time, and versioning of drawings (including differencing and merging) is far more difficult than versioning of text.
Like CMM and XP, MDA can't be adopted merely by the development team or merely by management. Both must be aware of the costs in time and tools when incorporating a new development process. The good news is that each of these methodologies offers improvements in the ability to predict, build, and release working software. If you're having a difficult time tracking your project, try using CMM. If you spend a lot of time reinventing the wheel, MDA may improve design and reuse. If your team spends a lot of time spinning its wheels, jump on XP.
Don't wait for a completely new project to use these methods. Early adopters within an organization can reasonably apply any new process to an existing project. Just remember that software process is a management issue, and it's management that must make sure that the process is being used properly and that corrections and improvements are made routinely. Is it free? Absolutely not. Is it worth it? Absolutely.
Bruce Levkoff is a software consultant who has been introducing structured design, object-oriented development, and agency-compliant software processes to companies for over 20 years. He can be reached at firstname.lastname@example.org.
- Beck, Kent. Extreme Programming Explained: Embrace Change. Reading, MA: Addison Wesley Longman, Inc., 2000.Back
- Demarco, Tom and Timothy Lister. Peopleware: Productive Projects and Teams, 2nd Edition. New York, New York: Dorset House Publishing, 1999.Back
XP has many interesting concepts and some good ideas. However, expecting the design to emerge during development (the opposite of MDA) makes it inappropriate for embedded systems. The XPers admit where the cost of change or defect is high, XP should not be used.
– Jeff Geisler
sr software engineer