How to use a debugger as a bug preventive tool - Embedded.com

How to use a debugger as a bug preventive tool

Generally speaking, a debugger is thought of as a tool to use after a problem is encountered, when its cause is not obvious. Although a good debugger is the best tool for diagnosing problems and defects in software products, their use in the code writing phase is often overlooked.

In this article, I will discuss how and why you would want to use a debugger before you know that you have a bug to track down and fix. At the end of the article, I will discuss some of the basic requirements that a debugger must meet in order to apply this technique.

Keep in mind that this approach is not intended to replace existing practices that also attempt to reduce bug rates. Testing and design are still important components of good software development. First though, some facts to frame the discussion:

1) The longer a bug is in a product, the harder and more expensive it is to fix. For example, if a developer discovers a bug in the code they have just written, it may take only moments to fix. If the bug is discovered later, the buggy behavior may actually be depended on other components that were written later. Fixing it may require major restructuring. Worse yet, if the bug is discovered after the product has shipped, it may require recalling the entire product. The sooner a bug is fixed the better.

2) Studies have shown that there can be as many as 1 bug for every 8 lines of code. Over time, testing will discover some of these, but others could languish in the code for a long time before they are discovered by an important user hitting an edge condition that wasn't fully tested.

3) Other studies have shown that 50% to 75% of the time spent on a typical software project is spent on testing and debugging. Additionally, the large amount of time spent makes it very hard to schedule projects.

4) Many software development methodologies attempt to address these issues by reducing the number of bugs that get built into the product in the first place.

We know that all code has bugs in it. The sooner you can eliminate these bugs, the fewer problems will need to be fixed in the testing and debugging phase and the sooner your project can be released.

So how does a debugger help you do this? Consider the following:

1) You know the most about your code immediately after you write it.

2) In any reasonably sized product, you can't really know how your code will interact with pre-existing code until you run it

3) In some cases your code will appear to work, when actually it is failing in a subtle way that you may not discover until later

Given these three points, we can conclude that the most opportune time to detect and repair bugs is immediately after you write the code. A good debugger can serve as a code inspection tool to help find the new bugs that have inevitably been added.

Debugging as Bug Prevention
The first thing you should do after you have written and compiled new code is bring it up under a debugger and step through each new line. It is important to do this before you try to run it “normally”.

The danger of running it outside of a debugger is that if it appears to work, there will be a strong temptation to move on to the next feature. Unfortunately, there are countless ways that bugs could have slipped through undetected.

So after writing code to add a new feature, it's time to bring it up under a debugger to see if it really does what you intended. To start with, set breakpoints on all of the new code. If you have added new functions, use the debugger to put breakpoints at the start of those functions. If you added new code to the middle of already existing functions, set breakpoints there, too. Now run the program from the debugger and attempt to exercise the new code.

It's surprising how often this results in… nothing. For some reason the code you added is not even being called. Perhaps you put the program in flash bank 2 but booted out of flash bank 1. Or maybe you misunderstood how your program parses commands, and you forgot to update the command table.

Whatever the reason, you're in a great position to start tracking down the problem ” it's already up under a debugger. In the case of the flash bank problem, you can verify that the new version of the program is running on the target. In the case of the command parsing problem, you can set a breakpoint on the command dispatch point to see how it works.

Let's say that you do start hitting your breakpoints. This is where things get much more interesting. At this point, you want to begin stepping through each source line of your program. You are looking for anything that does not match your expectations.

First of all, check that your code is being called in a way that you anticipated. For instance, is the call stack what you expected? Has the code been called before the program has properly finished initialization? Does the thread have the lock you assumed it would have? Are you even executing in the thread you were expecting to be in?

The easiest thing to check for, and probably the most important, is whether the arguments to the function you are in are the values you were expecting. This is also a good time to browse all references to the function you are viewing, in order to determine if there were any other call sites which may also need to be verified.

Looking at the variables
Next, look at the variables modified by your new code. Suppose you had a variable i which you expected to go from 0 through sizeof(myarray). Is myarray actually a pointer instead of a static array so i only goes to 4? Or perhaps you used <= instead of <, so you end up accessing an element off the end of your array.

Again, the best time to do this sort of analysis is right after you have written the code. You know exactly what that code is supposed to do, so it will take only seconds to see that it is not doing what you wanted, and then only seconds to correct the problem.

As you are stepping through your code, you will encounter branches that are not executed. Each time that you encounter such a branch, you should set a breakpoint on it for later verification. In this way, breakpoints serve as markers of unverified code. When you have no breakpoints left, you have verified all of your code, and you should have high confidence that it does what you think it is supposed to do.

In some cases, it can be difficult to generate real input to verify every branch. That's okay, though ” you're using a debugger, and the debugger has full control over everything in your program.

Every register, all memory, everything in your program can be changed, altered and otherwise manipulated to serve your verification ends. Thus you can reach one of these branch points and modify an important variable to “fake” a situation for which you otherwise may not be able to generate a test case.

For instance, take some code that is supposed to process input from a network. This input data should be trusted as little as possible. Even if there is no malicious hacker attempting to break into your system, someone else's buggy program could be sending you bad data.

By using the debugger to modify variables in your program, you can quickly verify that the program responds appropriately to corrupt data. Say it uses a lookup table with 1024 entries to determine the packet type from the first 4 bytes of data. Using the debugger, you could modify the memory address of the packet type field to be someing invalid, such as -1.

You can then step through your code to see how it handles the bogus data. If necessary, terminate the bad connection and go back to listening to new requests. Did you see your code free the memory for the packet? Did it remember to remove the reference to the network socket from the list of sockets to serve?

Using these techniques, it is fairly common to encounter a number of bugs in your new code. These bugs would probably have been found eventually, but the cost of discovering and fixing them right away is significantly lower than the days, weeks, or months they could otherwise linger.

Moreover, because you use the debugger to force values and pathways that do not happen in normal testing, the edge cases that only show up in the field under excessive load are actually tested.

In addition to fixing bugs sooner, there is also another advantage to this method of development. This process helps you gain valuable knowledge of the program being developed.

Many of today's programs are significantly larger than what a single person can comprehend. They can be millions or even tens of millions of lines of code, which when compiled are hundreds of megabytes in size. Working with codebases of this magnitude requires a constant balancing act, and most developers are doing it blindly.

Interrupts can fire in the middle of sections of code that were thought to be uninterruptable, functions that aren't reentrant safe can be reentered, innocent-looking function calls can result in deallocation of important data structures. All of these things are unexpected and will therefore not be anticipated in any new code.

However, if you carefully step through all of your new code right after you write it and take the time to thoroughly inspect the state of the system when your code is executing, you will notice things that would have otherwise slipped by, to manifest themselves as bugs much further down in the development process.

Debugging as Code Review
Along the lines of what has been discussed earlier, the debugger can also be used to prevent bugs when doing code reviews. The effects of any code modification can extend well beyond its surrounding lines.

This complexity can be nearly impossible to see if a code review is done as a simple textual difference between two versions. However, clever use of the debugger as a code inspection tool can allow a developer to see the larger implications of a change.

To do a code review with a debugger, set breakpoints on every block that has changed. When you hit those breakpoints, use the debugger to inspect the state of the system to help determine if the changes are valid. For instance, if the changed code assumes that it will not be interrupted, confirm that interrupts are indeed disabled.

Step through each of the lines that have been modified. This is very important because it forces you to actively pay attention to what has been changed. When reviewing code, it can be easy to skip over areas that you don't fully understand. Before you step from one line to the next, make sure that you really understand the code on that line.

This approach to code reviews forces you to be thorough, and at the same time makes it much easier to find problems. You don't need to stare at a text difference and mentally “compile” the program and “simulate” what it will do. Instead, you let the compiler compile it and the target run it, leaving you to focus on finding problems and their causes.

What you Need in a Debugger
Unfortunately, not every debugger is capable of doing the tasks outlined above. The following are some basic requirements for a debugger (Figure 1, below ) that can be used to implement this approach to development:

1) The debugger must be able to debug the language or languages you are using.

2) The debugger must load your application quickly; 10 seconds or less is reasonable. This is particularly important if you will be working with large applications, for which load times for some debuggers can be over 30 minutes.

3) Simple operations such as source level stepping, viewing variables and walking the callstack must be responsive. Anything more than a fraction of a second for any of these operations becomes tiresome very quickly when using this approach. Some debuggers take minutes for these operations on large applications.

4) The debugger must be able to read and write all registers and memory on your target. If it cannot, you will be unable to modify variables and try out different code paths.

5) The debugger must be able to dynamically make function calls, often called command line procedure calls. If you are unable to do this, you will be unable to test functions with different arguments.

Figure 1

Conclusion
Apart from fixing bugs sooner, there is another advantage to using a debugger early on in the development process: you gain knowledge and experience operating it. This means you know how to make it work for you when you need to track down the really hard problems. Think of a good debugger as a bug preventer, and use it early and often.

Nathan Field is an Engineering Manager for the MULTI Integrated Development Environment group at Green Hills Software, Inc. Nathan is a graduate of Harvey Mudd College and is keenly interested in tools that reduce the time and pain of the debugging phase of a software project. You can reach him at nathan_field@ghs.com .

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.