Transitioning from code-based to model driven software testing: Part 2 - Embedded.com

Transitioning from code-based to model driven software testing: Part 2

Having covered the basics of the UML 2.0 Software Test Profile in Part 1, here we will take a closer look at a sample design that was made abouttwo years ago using UML, but only recently has it been tested usingmethods that were mentioned above, based on the UML Testing profile.

The result was that a design that was believed to be correct wasfound to be non-compliant with one of the design's derivedrequirements. We will then follow with more advanced use of the UMLTesting profile, where existing Test cases and Test scripts need to beintegrated into the Model Driven Testing process.

Figure5: Requirements document (Textual)

Sample Design: An AM/FM Car Radio
Our sample design is a car AM/FM radio. The textual requirements,captured in a Word® document or in a Requirements Managementsolution like Telelogic DOORS, typically exhibits a structure similarto what is shown in Figure 5 above .

The use cases that need to be supported in this design are given in Figure 6 below.

Figure6: Main use cases for the radio

Now we will take a look at the design artifacts. Figure 7 below shows the ObjectModel Diagram of the Radio.

As one can see, it includes a Radio and Waveband classes and twoInterface classes. The one that we will test, the System Under Test(SUT) is the Radio class.

Figure7: Object Model Diagram for the radio

In this case, the behavior of the Radio has been defined usingStatecharts. While executing the design, one can look at the AnimatedStatechart, as shown in Figure 8 below .

Here, it reflects the status of the design right after switchingfrom Off to On mode, engaging the tuner, and then locking on a station,displaying the station frequency and defaulting to manual search mode.

Figure8: Animated Statechart (Brown: Previous state, Purple: Current state)

Now that we have looked at the Requirements, the resulting UseCases, and design artifacts (OMD and the behavior), we are ready to seehow the UML Testing profile can be used to test the Radio class.

Automatic Test ArchitectureGeneration
Normally, one would write code to develop the testbench (sometimescalled test harness) in which you can execute Test cases. In thecontext of the UML Testing profile, one needs to create a TestArchitecture, essentially a graphical model of code-based testbench.

The Test Architecture instantiate the SUT (the Radio class in thiscase) and Test Components (the Tuner, the Display and a User) that areneeded to interact with the SUT and the relationships between them. Amodern Model Driven Testing solution can generate these diagramsautomatically, based on the analysis of the design model. In Figure 9 below you can see such aTest Architecture.

Figure9: Automatically generated Test Architecture

Just like in an MDD environment, where code is generated from thedesign model, one can generate code form the Test Architecture. Andindeed, if one would generate code from a Test Architecture, it willcompile and run successfully, only that no Test cases will execute, andnone have been defined.

Normally, a Test plan will be developed based on the requirements.Then, Test cases will be developed according to this test plan.

Sequence Diagrams as Test Cases
Now that we have a Test plan, and a Test Architecture has beenautomatically generated, we are ready to create Test cases according tothe Test plan. Let's look at our first Test case. In Figure 10 below , we can see fourwindows, all related to a basic Test: “Make sure that the radio can beturned on and off.”

Figure10: Browser (including both design and Testing artifacts); Test casebehavior as Sequence Diagram; links to test plan and test executionreport

In Window 1 in Figure 10, you can see the browser that can be usedto browse through both the design artifacts and the Testing artifacts.

Window 2 shows the actual behavior of the Test case calledWB_TST001, captured as a sequence diagram. This sequence diagramconsists of instances of three Test components (the same that we havein the Test Architecture), and the SUT. It also describes theinteraction between these four components.

Initially, when the test starts, the SUT should clear the display.Then, pressing the evOnOff() button should cause the SUT to generate 4messages:

1. Send a blankAll() message(like an internal master-reset for the radio).
2. Display the waveband (inthis case, this is the value “3” standing for FM).
3. Tune to 87.5MHz.
4. Display the frequency(87.5MHz).

Then, pressing the evOnOff() one more time should turn off theradio, in this case, send another blankAll() message.

Window 3 shows the link from the browser to a description of arequirement that has been imported from the test plan (in this case, aword document). This is very important for traceability purposes.

Window 4 shows the report that has been generated based on theexecution of this Test case. Just like all other Testing artifacts, thereport, too, is accessible from the browser. And as one can see, inthis case, the test passed.

But, what does it mean that it passed the test? Executing the Testcase means that the classes that are described in the Test Architecturehave been instantiated: the TestComponentInstance of User did send anevent evOnOff(), the four messages that were described above were allobserved ” in the right sequence, and consumed by the correctTestComponentInstance. The evOnOff() has been generated a second time,and the blankAll() message to the TestComponentInstance of the Displayhas been observed as well.

In effect, the sequence diagram is a graphical test script, whichexecutes in the context of a Test Architecture. And since sequencediagrams are also used to capture requirements, one can re-use such arequirement sequence diagram and convert it into a Test case,effectively performing Requirement Based Testing.

Parameterized Sequence Diagrams asTest Cases
In the example above, we had to specify the “explicit” messages thatare being sent and observed. In order to leverage re-use, Test casesthat are described as sequence diagrams can also be parameterized.

For example, here is a Test case that turns the radio on, and thengoes up some frequencies, then goes down and checks for the wrap-aroundeffect once you get to the lowest frequency.

Describing such a Test case can be done by explicitly capturing allthe messages back and forth, but a more efficient way to do it is bycapturing a generic sequence diagram that accepts two parameters, whereone is up or down and the other is the required resulting frequency.

Calling the same sequence diagram several times with the rightparameters is a more efficient way of capturing a Test case such as theone shown in Figure 11 below.

Figure11: Test case based on parameterized sequence diagrams

Sequence Diagrams to Capture Stubsand their Behaviors
Another important aspect of capturing behaviors of Test cases is theability to define the behavior that is required from some of theTesting components so that the SUT will follow a certain behavior.Effectively, this will make Testing Components act as stubs with theirrequired behaviors. Let's take a look at Figure 12 below.

Figure12: Capturing test component behavior stubs (stubbing)

Here, we again have the SUT interacting with 3 instances ofTestComponents. However, this sequence diagrams has two parameters “freq1 and freq2. The behavior captured in this sequence diagram is onewhere the event evSearch(d=1) (i.e. search down) is being generated,and, as a result, the method isStrongSignal is being sent to the tunerinstance.

The tuner instance is a Test component and normally it has nobehavior. However, the sequence diagram specifies that the value thatshould be returned by the tuner is “0” (see the return value for thelabel of the message: “0=isStrongSignal()” ).

The same happens a second time, where again, we expect the tuner toreturn “0” for the second time, and only the third time return with a”1″.

The sequence diagram in Figure 12above defines not only the expected behavior of the SUT, butalso the required behaviors of the Testing components that interactwith the SUT for a particular test scenario.

In fact, looking at the code in Figure13 below shows that the code that is generated for the Testingcomponent (usually transparent to the user) is indeed intelligentenough to distinguish between the different calls.

Figure13: Test cases and stubbed operations

All the automatically generated code that is shown in Figure 13 isbased on the Test case in Figure 12, shown again in the bottom rightportion of Figure 13.

As one can see from the top left window, the operationisStrongSignal() is being used in two Test cases (SDWhiteBox_002 andSDWhiteBox_003) because the test component instance of the tuner isbeing used in both Test cases, and each of them may require a differentbehavior from the tuner.

Looking at the two calls that are highlighted, both are part of thesame Test case (SDWhite_Box_003, asin Figure 12 ), one can see the dispatching to the rightbehavior, the required return value as defined in the Test case, andits mapping to the generated code for the test component.

Next in Part 3: What to do when aTest fails.
To read Part 1, go to The basics of the UML2.0 Test Profile

Moshe S. Cohen is Senior Director, Telelogic. He holds an EE andMaster's in Mathematics and Computer Sciences from Beer-ShevaUniversity, Israel. He has over 20 years of experience in hardware,software and system design. From the beginning of his career, Mr. Cohenhas been applying formal methods and modeling solutions to design andtesting to develop robust systems and software. In his currentposition, Mr. Cohen is actively involved in defining Telelogic'stesting strategy and solutions.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.