Using requirements based testing to find defects in your software builds - Embedded.com

Using requirements based testing to find defects in your software builds

Our primary job, as QA and testing professionals,is to find defects in software builds. Fortunately, however, most of ushave moved beyond exposing and tracking bugs to the more critical roleof ensuring that the software our companies deliver meets customerexpectations before it is released. To do this, many organisations haverecently embracedrequirements-based testing (RBT).

Teams can be more effective using RBT because itenables them to target the root causes of quality issues by beingtightly coupled with application requirements and integrated throughvarious activities throughout the software lifecycle.

The challenge is to ensure these processes withexisting QA and testing best practices. To that end, these suggestionsto improve your RBT practices are offered.

Suggestion #1: test early andfrequently, so testing becomes a parallel activity to the developmentprocess, a constant pursuit that spans all roles and makes allstakeholders aware of quality objectives.

Suggestion #2: test with yourhead,not your gut, to inject repeatability and method in to the testplanning process, in order to make test coverage more predictable.

Suggestion #3 test withmeasurement and improvement in mind to quantify the status ofdeliverables and activities, so management can oversee qualityinitiatives across the IT application portfolio.

Figure1: The requirements-based software testing process flow

The quality crisis
The Standish Group's ChaosReport, and otherstudies, describes what has been the industry reality for decades; themajority of software projects fail to achieve schedule and budgetgoals. Poor software quality is a primary factor behind many failuresand often results in massive rework of application scope, design andcode.

Such rework extends release cycles and consumes significantadditional budget. To reduce software failures, it is imperative thatwe better understand the quality initiatives behind the products beingdeveloped for today's global economy. Industry experience and studiesshow the two primary reasons behind poor application quality aredefects in requirement specifications and problematic system testcoverage.

A study by James Martin shows that the root causesof 56% of all defects identified in software projects are introduced inthe requirements phase. Further, the study states that approximatelyhalf of those defects are the result of poorly written, ambiguous,unclear and incorrect requirements. The remaining defects can beattributed to incompleteness of specification; requirements that weresimply omitted.

When quality is handled as an afterthought, qualityverification efforts typically begin only once code has been completed.At this point, the testing team is under the gun to certify theapplication as quickly as possible and such certification is oftenperceived as a bottleneck that prevents deployment into production.

Inthis environment, it is not only difficult to ensure that therequirements are correct, but also to properly plan tests, to ensurecorrectness, completeness and solid coverage of requirements and togain visibility into the various quality aspects of the testedapplication.

It is no wonder this frequently proves to be acostly and frustrating exercise to all involved.

In addition, change management has become unwieldyin many organisations with application requirements changing hourly ordaily. However without the ability to properly manage these changes,traceability between requirements and test cases is often lost, makingit even harder to ensure continuous test coverage.

RBT is an approach implemented through well-definedprocesses, which target the root causes of the quality problems. Evenif your organisation is not using the practice today, most of us arefamiliar with the key processes within RBT, highlighted in Figure1 above .

RBT best practices
Defects detected early in development phases arecheaper to fix and result in significantly fewer surprises when thecode is actually delivered. Make testing a parallel activity to thedevelopment process to make all stakeholders aware of qualityobjectives.

Other 'best practices' include the validation ofrequirements against business objectives. This optimises project scopeby ensuring that each requirement satisfies at least one businessobjective. Review of requirements by stakeholders should also be usedto refine the requirements before additional work is done.

Today, it is uncommon that test case design isperformed in a systematic and rigorous manner. Rather, many tests aredesigned based on gut feel and intuition, which leads to unpredictablequality. Best practices within RBT require organisations to supportmethodical and systematic test case design to ensure predictable testcoverage.

Using RBT best practices, organisations can manageand improve quality processes through measurement. Throughout the RBTprocess, multiple measures can be used to quantify the status ofdeliverables and activities. This helps managers and process expertsoversee quality initiatives across the IT application portfolio.

Traceability also plays a critical role fororganisations using RBT since maintaining traceability informationbetween requirements and test-cases and tests is crucial. Thisinformation is required for monitoring progress and coverage, as wellas properly managing the impact of changes in requirements. Without it,it is more difficult to determine which test cases or tests should bechanged if a specific requirement changes.

While capturing and maintaining traceability datais very important, many software development organisations find that itis extremely difficult to get right. The primary reason is because mosttools in the market require the individual practitioners in thesoftware delivery chain to manually create and manage traceabilityinformation. This is an overhead that many are not willing to accept.To deal with this challenge, strongly consider tools that canautomatically manage links between work products.

By deploying solid RBT and QA/testing bestpractices, organisations will find they are better equipped to maximisethe business value of their software through quality initiatives thatspan the complete software lifecycle.

Moty Aharonovitz is Senior Directorof Product Strategy at BorlandSoftware.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.