Doing C code unit testing on a shoestring: Part 2- Code coverage analysis

Assume for a moment that all of the functions in your program arewritten using only if/else statements to control program flow. How doyou test it?

A test achieves 100% code coverage if and only if every if statementwas executed and at least once the condition was false and at leastonce the condition was true (somepeople call this “branch coverage” ).

If you always strive for 100% coverage (as you should), the first conditioncan be dropped: A test achieves 100% code coverage if and only if thefollowing holds: If an if statement was executed then at least once thecondition was false and at least once the condition was true. This canbe easily proved by induction on block nestedness level.

If your INSTRUM_IF macro's output uniquely identifies the specific if statement then it ispretty straightforward to analyze the overall test output to see ifevery if statement reported its condition both true and false at leastonce. Otherwise, a more involved instrumentation is necessary.

Instrumenting the whilestatements
The whilekeyword can be replaced in a similar manner. We put in instrum.h:

# ifdef INSTRUM_WHILE
# definewhile(condition) INSTRUM_WHILE(condition)
#endif

<>And of course a mutatis mutandisentry is added to the instrumentation specific header such as
instrum_common.h.

extern intinstrum_while(constchar *condition_name,
                                     int condition,
                                     const char *filename,
                                     int line,
                                     const char *function_name);

#define INSTRUM_WHILE(condition)
while (instrum_while(#condition,(condition)!=0,
__FILE__, __LINE__, __FUNCTION__))

An implementation of the function instrum_while may, however, have apeculiarity. There are two idioms, while(1)and while(0)which you maywant to treat differently. The first one is a synonym for for(;;) whichI don't particularly like but the fact is that some reputable people douse it.

The second one is a part of the do{…}while(0)construct which iscommonly used on two occasions: in macro definitions to wrap a blockand straight in the code to make the break statements do the work ofthe dreaded goto[end of block] .

You might want to instrument while(1)asfor(;;)and not instrumentwhile(0)at all. Here is a simple implementation of not instrumentingthese idioms:

intinstrum_while(constchar *condition_name,
                          intcondition,
                          const char*filename,
                          int line,
                          const char*function_name)
{
    if(strcmp(condition_name,”0″) != 0&&
        strcmp(condition_name, “1”) != 0){
            printf(“Loopcond. %s in function %s”
                   ” (file %s line %d) is %sn”,
               condition_name, function_name,
               filename, line,(condition)?”true”:”false”);
    }
    return condition;
}

Of course an implementation like this should be kept in a file whichis not instrumented.

Instrumenting the switchstatements
If the goal of instrumentation is simply to announce the value of thecontrolling expression of a switchstatement, we can follow the patternestablished above and place

#ifdef INSTRUM_SWITCH
# define   switch (ctl_stmt)INSTRUM_SWITCH(ctl_stmt)
#endif

ininstrum.h etc .

This would be good enough for establishing a regression base andusing it in regression tests. However, if we want to prove codecoverage, we need information on whether a given caseor defaultwashit. That is to say, we need to instrument the caseand defaultlabels,which is our next subject.

Instrumenting the defaultstatements
To instrument default, let's follow our usual pattern and put ininstrum.h

#ifdef INSTRUM_DEFAULT
# definedefault INSTRUM_DEFAULT
#endif

Now, to sensibly define INSTRUM_DEFAULT,we need to use the defaultkeyword and to stick instrumentation codesomewhere around. To do so, observe that if falseval evaluates to 0,then

default:

in any context is functionally equivalent to

default:if (!falseval) a_unique_label:

and to

default:if (falseval) ; else a_unique_label:

Between the two I don't have a preference. (A unique label will be needed to consume adangling colon. It will remain unused, and most compilers will issuewarnings about unused labels. You can safely ignore or suppress themfor the unit under test.)

For falseval we can take

(instrum_default(__FILE__,__LINE__,__FUNCTION__),
!instrum_false)

where
extern void instrum_default(constchar *filename,
                                            int line,                                             const char *function_name);

is some function that e.g. announces hitting a default statement,and

externint instrum_false;

is a variable with the value 0 in a different translation unit (so that the compiler is not tempted tooptimize anything out). Note that because of the comma operator,the whole expression for falseval must be parenthesized (because if is alreadya macro!)

For a unique label we can take a concatenation of the wordinstrum_label and the line number (e.g. a unique label for line 2007will be instrum_label2007 . Thisis a common C fare –

CAT(instrum_label,__LINE__)

where the CAT macro concatenates the two expanded arguments:

#define CAT1(a,b) a ## b
#define CAT(a,b) CAT1(a,b)

NOTE. You may choose to run your (instrumented) tests in the hostenvironment when possible. If you use Microsoft Visual C++ to build theexecutable, the construction of the artificial label “CAT(instrum_label_, __LINE__) ” may be broken because of the brokenimplementation of __LINE__. To fix this, you can remove the support for”Edit and Continue” (command-line option /ZI) in the project, or, forversion 7.0 and above, use the non-standard __COUNTER__ instead of__LINE__. Many thanks to Alf P. Steinbach for pointing it out.(Of course, other compilers may have their idiosyncrasies, too.)

Now we are in a position to put the pieces together and to define INSTRUM_DEFAULTas follows:

#define INSTRUM_DEFAULT
        default :         if ((instrum_default(__FILE__,__LINE__,__FUNCTION__),
               !instrum_false))
               CAT(instrum_label, __LINE__)

For this instrumentation to compile there cannot be two (or more)defaultlabels on the same line (or else we'll produce two identicalartificial labels). But for this to happen, there must be two or moreswitchstatements on the same line, which is not a terribly goodpractice; we can ignore it. After all, if you do engage in thispractice, a compilation error will notify you.

Instrumenting case statements:help from a coding style needed
Our next step is to instrument the case labels.That is, we are goingto create a macro case .For such a macro to be useful, we must somehowmake the numeric label itself, however indirectly, to participate inthe macro expansion. The only sensible way of achieving it that I foundis to pass the label as a parameter to the macro. In other words, wewant something like this:

#definecase (x) case (x):SOMETHING(x)

<>With this macro, consider
        case MYCASE:
       case 2007:
       case (HERCASE):
       case (HISCASE):

The first two occurrences of are not recognized as macro calls(missing parameter list) and are not replaced. The third and the fourthoccurrences are valid macro calls and will be expanded as desired.

Looking at this example from the vantage point of writing the code,we can conclude that for a caseto end up being instrumented, itsnumeric label must be parenthesized. This, admittedly, is not a commonpractice. But that's how you need to write it to instrument thestatements in order to prove code coverage. This is a matter of yourteam's coding standard.

Having resigned to instrumenting only caseswith parenthesizedlabels, let's put a macro implementation together. Following thepattern, put in instrum.h:

#ifdef INSTRUM_CASE
#    define case (x) INSTRUM_CASE(x) #endif

For the implementation of INSTRUM_CASE in instrum_common.h,observe that, just like for default, a passage

case (x):

is functionally equivalent to

case (x):if (!falseval)a_unique_label:

Following the same pattern as for default, we can do the following:

#define INSTRUM_CASE(x)
        case (x): if((instrum_case(#x,x,
__FILE__, __LINE__, __FUNCTION__),!instrum_false))
INSTRUM_CAT(instrum_label, __LINE__)

where instrum_case is an appropriately defined function. The type ofthe second parameter must be large enough to hold any label used inyour application's switch statements. Hopefully, long long will do, ifyour compiler provides it.

It is not uncommon to see several cases in a single line if theyhave common implementation, like

case (HERCASE): case (HISCASE):

The instrumented code won't compile because of a duplicatedefinition of the artificial label. Again, if you want to use thisinstrumentation, make it a rule to place a case in its own line. If youdon't, the compiler will notify you.Instrumenting the for statement
The syntax of the forstatement makes it difficult to invent an abusiveinstrumentation macro replacement in a generic way. It is not easy toget to the loop control expression which would be the key: recall thatthe purpose is to ensure that the control expression was at least oncetrue and at least once false. About as much as one can do is this:

#ifdef     INSTRUM_FOR#define     for(triplet)INSTRUM_FOR(triplet)
#endif

in instrum.h ,and in instrum_common.h put

#define INSTRUM_FOR(triplet)
        for (triplet)
           if ((instrum_for(__FILE__,
           __LINE__,__FUNCTION__),instrum_false)) ;
           else

Explanation of this macro goes exactly as that for INSTRUM_DEFAULTand is omitted here.

This instrumentation may be useful for regression testing. To makethe instrumentation more useful for proof of code coverage, we need toresort to the help of the coding policy. We can require thatinstrumentable for statements have macro-ized controlling expressions,i.e., instead of writing

for (expr1;expr2; expr3)

we write

for (expr1;ISTRUE(expr2); expr3)

where the non-instrumented ISTRUE macro isdefined as identity macro, i.e. instrum.h has

#ifndefISTRUE
#define ISTRUE(e) e#endif

For example, an instrumented version in instrum_common.his 

#define ISTRUE(e)
instrum_istrue(#e,(e),__FILE__,__LINE__,__FUNCTION__)

with an appropriate definition of the function instrum_istrue.

This affects the coding style even more intrusively thaninstrumentation of the caselabels, and therefore may affect yourdecision on whether to use this instrumentation or not.

Instrumenting the breakstatements and others
There is nothing interesting to learn about a break statement otherthan that it was executed. A definition like this will do:

instrum.h
#ifdef INSTRUM_BREAK
#    define break INSTRUM_BREAK
#endif

instrum_common.h #define INSTRUM_BREAK
        if ((instrum_break(__FILE__,__LINE__,__FUNCTION__),        instrum_false)); else break

The purpose of the function instrum_break is merely to announce theexecution of the correspondingbreak statement.

The explanation of this scheme exactly follows that of default, andis based on the observation that

break

in any context is functionally equivalent to

if (falseval); else break

It should be noted that instrumenting the break (and continue)statements adds nothing to code coverage analysis: each is the lastexecutable statement in a conditional branch (or you have dead code which your compiler” or at least your Lint ” will tell you about).

So instrumenting the condition evaluation (and, for a break in aswitch, thecorrespondingcase ) provides all the information needed. Thesame argument applies to return and goto statements.

That said, your coding policy may prohibit using goto,and maybe,even continue.In this case, you may want to instrument them to makethe test case fail.

Still, all of continue,goto, return can be instrumented using thesame scheme as break. Note though that instrumenting the returnhas alittle peculiarity: copying thebreak instrumentation one to one willhave the apparent effect in an instrumented function of an executionpath without a return. This will (or at least could) elicit thecompiler diagnostic. To correct this, use an endless loop in the falsepath, like

#define INSTRUM_RETURN
        if ((instrum_return(__FILE__,__LINE__,__FUNCTION__),
       instrum_false)) {for (;;);} else break

Next in Part 3: Putting it alltogether
To read Part 1, go to “Unit testingrequirements and what you will need.”

Ark Khasin, PhD, is with MacroExpressions. He can be contacted atakhasin@macroexpressions.com.