Ensuring software quality & reliability with configuration & change management

Configuration management is a fundamental product development process.In fact, we would say “you don't knowwhat you are doing if you don'tknow what you have; ” that is, you don't have much of anorganization ifyou don't know what your product is or the resources that areavailable.

We know from experience in various companies that an optimallyrunning configuration management program will reduce erroneous faultreports and will also stream line test times. Manufacturing firms withgood configuration management systems bring their part numbers andproduct releases under control.

In general, the enterprise is completely dependent on robustconfiguration management, whether the product is software, firmware, orhardware. Sending the customer the incorrect version of a product is aquick way to become a “de-sourced” supplier.

The Purpose of ConfigurationManagement
Configuration management systems help to deliver coordinated softwareand hardware. They do this by easily identifying features andfunctional content within a particular software and hardwareavailability.

Not only that, but we can have straightforward traceability offunction and feature growth over specification revisions as well asover content–hardware and software–delivery. Throughout the process (Figure 1, below ), we can alwaysclearly identify changes, execute pre-release containment, and identifycompatible components within a system.

Figure1. Configuration Management Process

Configuration management andplanning
In order to deliver robust software or hardware, it is necessary thatwe plan and coordinate the software and the hardware deliveries offunctionality.

In short, functionality, deliveries, testing, and production followa master plan (Figure 2, below )rather than ad hoc pandemonium. With appropriate planning we willalways know the following items:

* Features and feature revision level
* For example, engine electronic control unit, cruise control rev 1.0within a particular software revision
* Software revision level
* Hardware revision level
* Subassembly revision level in the case where we have firmware'burned' onto a chip
* Systems integration (phased feature release)

Figure2. A configuration management mast plan for hardware and software

IEEE standard 1220 favors the master plan approach to systemengineering, providing an alternative to standard project managementpractices.

Configuration management areas
Configuration management can be applied to specifications(documentation), models, simulation, hardware, software, thecombination of hardware and software, and to system integration.

That truth is that configuration management can be applied the justabout anything. In a manufacturing facility, we might see the followingapproach to Configuration Management, namely:

* Product baseline
* Product change management
* Number of product changes
* Time spent on product changes
* Cost impact of changes
* Product configuration (CMP)
* Test configurations (test coverage)
* Configuration audit

The items in the list apply to both software and hardware for themost part.

The 4 Department of Defensecomponents
Military standard 973 and Military Handbook 61 spell our fourrequirements for any configuration management system: 1) configuration identification; 2) configuration control; 3) configuration status accounting;and 4) Configuration auditing.

Configuration identification
To establish configuration identification, we must have the means todefine a configuration item. In DoD software implementation, these areknown as computer system configuration items (CSCIs or 'siskies').

Generally, we pick an item and define the initial version of it as abaseline. This base-line will function very much as a base-line in aproject management tool; namely, it allows us to know where we startedand to detect and record change as the project or product developmentproceeds.

In many cases, we will define a part number, describe the featurecontent, apply a specific function and revision level, and we may evendefine further level of refinement. At every stage, configurationidentification provides a process and product that is document-able andcontrollable.

Configuration control
Configuration control is the part of configuration management where wesupervise changes to the configuration of the product, whether it issoftware or hardware.

In some cases, we may re-baseline the product. In general, weusually have only one version in release at a time. Control can becomecomplicated during transitions when more than one version may beavailable; in these cases, we can control with part numbers.

Management can also become complicated with multiple componentconfigurations for a system, particularly in the age of masscustomization of products. With motor vehicles, we often seeattribute-based requirement or line sequencing, in which the parts arecustomized to the vehicle identification number (itself a form ofconfiguration identification).

This production complication has ramifications to the verificationactivities. Specifically, these production variations have to beaccounted for in the testing activities.

Typical configuration control baselines might be the following:

* AllocatedBaseline (ABL) . The initially approved allocated configurationdocumentation which is often allocated from a higher level or system.

* FunctionalBaseline (FBL) . The approved functional configurationdocumentation that adds verification to the mix.

* ProductBaseline (PBL) . The approved product configuration documentationwhich generally includes a description of the acceptance testing andmay also include the actual hardware and/or software.

In all cases, a configuration baseline involves an agreed upondescription of the attributes of a product, at some point, whichbecomes the starting point for defining change.

There should be an approved and released document or documents(revision) for every specific version, the purpose of which is toprovide a solid foundation for managing change.

These baselines always include the currently approved and releasedconfiguration documentation and, with software, a released set of filesmaking up a software version and related configuration documentation.

Configuration status accounting(CSA)
Configuration status accounting is used primarily for the reportingfeature, although its significance is much broader. Under CSA, werecord changes and update configurations when items change and we issuereports.

When dealing with software we might audit the frequency ofcheck-in/check-out to verify that the developers are protectingintellectual property. We may audit the release documentation toconfirm that the proposed or planned configuration has been delivered.Additionally, we might take a look at the count of changes to get anidea of the stability of a given release.

CSA can be supported by software (for example, the open sourceproduct Subversion) and it is definitely a component of product datamanagement software, product lifecycle management software, anddocument control packages. However, it is also possible to useubiquitous spreadsheet software.

Figure3. A very simple configuration management process flow

Configuration auditing
An important part of the configuration management process flow (Figure3, below), configuration auditing has two primary actions: physicalconfiguration and functional configuration, both of which will comparethe configuration expected to what is delivered.

A physical configuration audit compares existing documents tocontracted or required documents and a functional configuration auditverifies functionality against requirements. The results are part ofthe release notes.

Hardware Configuration
Hardware configuration management is most commonly accomplished throughthe part number system and the associated bill of materials. The billsthemselves can take on different structures; for example, modular,phantom, or one-level bills. It is not unreasonable to ask that thebills of materials themselves fall under configuration managementcontrol.

Software Configuration
With software, we usually are well-served using dedicated softwareconfiguration management tools; for example:

* Subversion
* Revision control system (RCS)
* Concurrent version system (CVS)

All of these programs lock out other users from making changesduring an edit and provide some level of reporting. They basicallymechanize the software build process by defining what modules areincluded and what revision of the modules. Most 'make' programs have ameans of accessing the software configuration management systemautomatically.

Hardware and Software Firmware
When we have a combination of hardware and software, known as'firmware,' we need to have coordination with these embedded projects.The situation is set up as an assembly part number with twosubassemblies: the chip and the software—together they make thesubassembly. For example:

* Subassembly X104AG573
* Product Y508BBD99 Rev 4.2 software
* Flash Microcontroller HX349855S

Configuration and Verification
Any testing performed on a configuration must be identified explicitlywith the following information:

* Hardware release including subassemblies
* Software version
* Developmental version or
* Released product version (they are not necessarily the same)

The configuration items should be linked to the test cases andtest documents. Of course, the documents themselves can also come underconfiguration management.

Release notes
Release notes are linked to the hardware configuration by part numberwith the hardware content and known bugs identified. The release notesare also linked to software, again by part number, revision levels,software modules, functions / features, and known bugs

Change management
Change management is an umbrella term that contains configurationmanagement. For example, change can occur to a process that is actuallya service and not a tangible product. Regardless, we still have toconsider scope and configuration impacts.

If we are on a project, we want the project baseline to bewell-defined at project start, particularly with respect to the processand the related areas of responsibility. To help with this goal, wesuggest creating a formal configuration management plan.

The requirements themselves should fall under this plan.Furthermore, all changes should receive cost and time quotations and beagreed upon by all parties.

Configuration management andrevision control
For software, the tool should allow branching, merging, developmentalrelease numbers, and released product release numbers (not same asdevelopmental). Sometimes, customers prefer to designate the releasenumber instead of using the developmental release version number. Thatmeans the software configuration management software must be able tosupport multiple numeration schemes.

Managing released products
Released products come under the same controls as developmentalproducts. They need to control the engineering release documentation,design change notifications, engineering change requests, and anysimilar documentation.

Any automotive, commercial vehicle, or ISO-certified organizationwill have configuration management for released products to avoidcontaminating the supply chain with nonconforming material.


Configuration management andoutsourced work
If anything, configuration management rises in importance whencontracting outsourced work, particularly with software development. Ingeneral, the more distributed the development team, the moreconfiguration management is a priority. We can use technology to helpmaintain document and item integrity. The distributed team can useexisting software configuration management tools as well as data basesand/or Microsoft Sharepoint.

Configuration and systemsdevelopment
With systems development, we will have multiple configurations makingup a system configuration and multiple “releases” of configurations.Many organizations deliver the functions of the system in stages, withcertain features in each component that comprise the system.

This extends to items such as customizable product parameters(for example, those stored in non-volatile memory) alter the productfeatures to customer demand, increasing adaptability and customercontentment. All of this requires updates to the system configurationand tracking of the revision progression. Under these conditions, wewould expect more branching and less merging of configurations.

Deviations are a written authorization granted before manufacturing aproduct which allows us to release the product from meeting one or morerequirements. These documents only work if they represent a temporarycondition that is controlled by time or by count of product. They arean explicit recognition of temporary modifications outside bounds ofchange control system, although a deviation is, in a sense, part ofthat system.

As with all forms of configuration management, deviations requirean identification method such as serialization, part number control orsome other marking. Deviations not adequately accounted for andcommunicated, can cause great harm to a project and product developmentand verification.

Configuration failures
When the system breaks down, we can see non-functional components andsystems. Testing may be come inefficient with reporting of faults thatare not valid. We may also miss expected functions as well as seeduplication of cost in order to test again.

We have also seen situations where a supplier shipped prototypeparts with an unknown hardware version and an unknown softwareversion—the product didn't work on arrival at the customer site and thesupplier later lost the business.

We say that configuration management is indispensable to deliveringquality products. Failure to manage the configuration leads to sendingthe wrong item to a customer or fixing the wrong version of softwareand illustrates a lack of part number control; any these evils can becatastrophic to a customer/supplier relationship. Good configurationmanagement always leaves a competent audit trail.

Kim H. Pries is Director of Product Integrity andReliability for Stoneridge, Incorporated; specifically, StoneridgeElectronics-North America. He is responsible for all test andevaluation activities including laboratory, calibration,hardware-in-the-loop software testing, and automated test equipment.Kim wrote a Six Sigma book and, with co-author Jon M. Quigley, a bookon project management of complex and embedded systems.

Jon M. Quigley is the Manager of Verification and Testgroup at Volvo 3P electrical in North America. Jon is responsible forverification activities ranging from component, sub-system and system(hardware in the loop simulation), including environmental and EMCrelated requirements on the product. He is also responsible fordevelopment process improvement within the Verification group andinterfaces to other parts of the company, including embedded productdesign and aftermarket. Jon is the co-author of a book on projectmanagement of complex and embedded systems.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.