Developers at network equipment manufacturers (NEMs) are busier than ever creating complex devices with impressive features—and countless testing requirements. These days QA test time can take twice as long as development. Add in time for acceptance testing by service providers and the quality process can swell to more than two years.
That's two years of labor costs and missed revenue opportunities. Still, testing cycles appear to be growing, along with time-to-market and testing team size. To stop this trend and deliver higher quality products to market faster, companies need to start getting developers more involved in the testing process.
This is easier said than done.
Many developers believe that more time spent testing means less time spent developing new features. But actually, the opposite is true. By not doing more testing, developers risk having bugs escape to QA or even to customers, where they are far more costly and time-consuming to fix.
Clearly, testing should be an integral part of development, but how can we make this work for both developers and testers? The answer is to find an effective way for developers to test and still remain focused on development.
Why developers need a better way to test
Testing for developers used to be limited to white-box test design. Then it grew to include black-box test design, with developers validating feature interoperability at the code and device levels.
Now two unrelated forces are driving developers to test even more: pressure by NEMs to deliver higher quality products to market faster, and the widespread adoption of agile and XP programming.
The volume of testing, however, is not nearly as taxing as the testing process itself. Today, developers do most black-box and feature interoperability testing manually, or testers write code to do it. Either way, this is a time-consuming process, since unit testing must be done for each check-in by each developer and for each build before it goes to QA (as a quick sanity check).
Developers also face challenges with the build regression system, a set of test cases that run each time a new build is checked in. Developers use this to determine whether the build is solid and ready for QA. (Nothing is worse than sending a build to QA only to have 50 bugs found instantly, starting with the install failing.)
Yet, like unit testing, this system is manual and built on scripts and code. In addition, it is difficult to maintain, update, and coordinate.
If developers had a simple tool that could help them do unit tests automatically, they would have more time to code new features. Likewise, if developers could feed these automated feature tests into an automated build regression system, they would save both time and resources.
What to look for in automated testing tools
To streamline testing, developers need a way to easily reproduce and then solve bugs. And like the automation team, they also need to pay attention to the integration of all the systems they must now touch in the testing cycle.
Defect tracking, build systems, version control, and even equipment reservation software require a workflow and some integration to achieve the highest quality feature velocity with the least wasted effort.
Automation by itself, though, is not the answer. Developers need automated testing tools that not only allow them to automate key tasks, but also communicate clearly and create assets that can be easily shared and maintained.
This will save them time, allow them to employ new agile and XP programming methodologies, and add a new level of efficiency to the entire quality process. An automated testing tool that generates clear, detailed documentation can help developers overcome many of the most frustrating and time-intensive challenges of testing.
For example, reproducing a QA-reported problem or bug is usually half the battle for developers. If they have a replayable test case that can replicate the same actions to see the bug, half of their job is done. As for the other half, they can resolve the issue two to five times faster—and validate the fix.
Each time developers unit test, they usually must script, code, or build a harness. Automated testing tools can eliminate redundancy and streamline processes, enabling developers to test faster and run the same test case for each build.
In addition, these feature tests can run at night or while developers code another feature, and allow them to easily see whether the features operated correctly. Developers might have to invest a little time learning a new tool, but they stand to benefit from significant time savings and greater productivity.
Picking the right tools
With the right automated testing tools, developers can increase efficiency throughout the quality process by creating test assets that both testers and developers can leverage. These test assets can help QA teams quickly understand which feature aspects need to be tested and how they should work, eliminating confusion and the potential for missed bugs.
And though unit test cases will likely never make it to the regression system, leveraging some of the automation team's assets can be a huge time saver for developers. If they can grab a core test scenario or key procedure, or leverage a response map, it makes their job easier and faster.
They no longer have to spend time trying to adapt the test to the build system, managing and updating the test for new builds, or trying to figure out someone else's code or script.
Being able to leverage these assets also makes tests more maintainable. Developers usually run the same unit tests until a version ships. They then create new unit tests as they develop the next set of features.
However, if the test can leverage assets maintained by the automation team, then build regression test suites will continue to grow, driving up the quality of builds that go to the QA team.
Many of these assets are leveraged behind the scenes, but they require clear ownership to maximize their value across teams and the quality lifecycle. For example, the automation team needs to have someone be responsible for updating the assets, and another person to serve as a liaison to development. The development team should do the same.
That way, if engineering knows that a new feature changes a response or output, they can inform the automation liaison. The change can then be incorporated so that development and the entire quality chain can leverage it, reducing work for everyone.
Achieving better quality and greater efficiency
Businesses can benefit in key ways by having developers more effectively participate in the quality process. Developers who can more easily create and maintain unit and build regression tests can do more testing in less time, resulting in greater testing coverage and solution quality.
And with a testing tool that provides a communication platform, developers can easily communicate and share test assets with all involved in the quality process, reducing costs and time to market. In this way, providing developers with automated testing tools can increase efficiency throughout the QA process and yield a significant competitive advantage for the company.
David Gehringer is vice president of marketing at Fanfare Software. He has bachelor's degrees in mechanical engineering and aeronautical engineering, both from University of California, Davis.