How does scrum framework work in an embedded systems programming environment?
My first post advocated adopting an agile approach to embedded software development. One agile principle is breaking up projects that are over a month or so in duration into a series of smaller deliveries. These deliveries might be suitable for a customer to assess or even actively use, or might be intended only for internal validation. They allow progress to be measured by demonstrating working code and allow closer customer collaboration to verify that the product will meet their need. Breaking functionality across iterations focuses the team on the immediate delivery and allows changes to be accommodated in future iterations with relative ease.
The scrum project management framework is a well-proven approach for iterative development. In scrum, the work to be undertaken is divided into smaller pieces, preferably as user stories capturing a user-testable feature, but often as a task listing derived from a work breakdown structure . These work items are prioritized in a backlog . Items to be completed are selected at the start of each iteration (aka, sprint ), highest priority first. Sprint lengths are preferably fixed and short; four weeks or less is common.
Scrum recommends that each sprint deliver working software. This is considered impractical by many teams, putting some off of scrum completely. However working software does not have to mean delivery of a fully functional release, or even the minimum viable product. And in an embedded environment working software in early iterations may well rely on mocks and stubs of yet to be delivered hardware.
IEC 62304, an ISO standard for medical device software lifecycle processes, notes three feasible development strategies:
• Waterfall. Development proceeds once through a single development cycle.
• Incremental. System requirements are defined in full up front, delivery of these requirements is then broken in to a sequence of iterations.
• Evolutionary. System requirements are only partially defined up front; each iteration includes requirement refinement work.
For this post I'll provide an example of a project where we've followed an incremental strategy, and how Scrum was a good fit (Scrum also fits naturally within an evolutionary approach, but that's for another post).
The customer in this case, a developer of medical diagnostic devices, has an existing product with a mix of data sources and targets connecting over a variety of hardware interfaces. The data is subject to a number of data processing algorithms and the results are displayed on a touch screen. A complete software re-architecture was required to enable the device to support a number of new data sources and to make it possible to use a single codebase on multiple new platforms and operating systems.
The requirements, hazard analysis, and safety cases were already in place for the existing system, so the scope for the rebuild was well understood. Stakeholders, naturally cautious of an entire redesign, needed early assurance that the new software would deliver the improvements and flexibility required.
The overall development was allocated to a number of monthly iterations, with common (highly depended upon) and hardware-agnostic code scheduled for early sprints. An initial sprint planning meeting is held for each iteration, where the features to be delivered are agreed and captured in a backlog. This listing, commonly captured in a spreadsheet, is a living document visible to the team and stakeholders throughout.
Some features are labelled low priority for a particular iteration and only implemented if time allows. The allocation of features to be delivered in future sprints is only very roughly planned. Where features are yet to be started the customer can swap in alternates or change priorities as needed.
Short (15 minute) daily scrum meetings are held where each team member summarizes the work that he or she has done since the last meeting and intends to complete before the next. Any blocking issues are raised also. The customer provided a product owner who attends all scrum meetings; his role is crucial in agreeing feature details or advising on prioritizations, and he has excellent visibility of progress. Care needs to be taken to ensure that the scrum meeting does not become a daily design meeting; such discussions should be kept separate as they rarely require the whole team.
In the first month, we delivered architectural services implementing cross cutting concerns such as persistence, fault management, logging, and an operating-system abstraction layer. Delivery in this context means that the software features were designed with the design documented in UML and text, production code and unit tests were written and reviewed, unit-testing coverage was measured on all target operating systems, and Doxygen was used to provide API and test-code documentation. By the end of the iteration the customer had seen all these aspects demonstrated, and a continuous integration server (we use Jenkins with CxxTest) gives everyone visibility that the code compiles and all tests pass whenever the code base changes.
For the second month's iteration, we ported the existing data-processing algorithms to the new architecture. Rather than testing these against data passed over hardware interfaces, we wrote production-quality import/export modules to provide the algorithms with data read from prerecorded data logs and to write the results to output files. Such modules are invaluable for providing unit tests for the algorithms; they enable the processing to be tested before hardware drivers or user interfaces are available. We also found the modules to be valuable for adding a demonstration mode and enhanced debug to the production system. Delivery of these features was as for the previous iteration, with the addition of automated integration tests demonstrating the interaction of multiple modules. As the software had been written in an OS-agnostic manner, the customer could run these integration tests on their desktop PCs. This let them perform data processing across a variety of data files recorded from their existing system and confirm for themselves that the results were as expected.
The third iteration saw the implementation of the hardware data interfaces importing and exporting data from the system. This was the first iteration where it was necessary to use the target hardware, and the delivery of this iteration was the first time that the customer, who for this project owned system level validation, could start their validation activities. All hardware drivers were developed in isolation with their own dedicated unit tests before being integration tested within the wider system.
We're now in the fourth iteration and we've moved from porting existing data sources to adding new hardware devices not previously used within the existing system. The customer gained flexibility in having an extra quarter to finalize the requirements for this new functionality, and we're working closely with the product owner to refine the requirements, safety cases, and so on, as we learn more about the new devices. Future iterations will introduce the new user interface and higher level application services. The customer, having full oversight, can choose which deliveries will be targeted for release and hence schedule formal validation, instigate change management, etc.
To summarize, by building modular components and managing dependencies on hardware, we have been able to deliver working software in early iterations followed by incremental introduction of software and hardware features. Our customer has really appreciated the level of visibility and collaboration that this iterative and collaborative approach allows, and they have been surprised at how quickly they have been delivered a working solution that they can validate themselves.
Mike Hogg leads an embedded development team within Zuhlke, an engineering consultancy. He and his colleagues have been using agile techniques and their precursors for over a decade in multiple sectors including telecoms, transport, and medical. He has 15 years development experience primarily in real-time embedded systems written in C++ and C, and he has also delivered business applications in Java. Mike writes the blog Agile Adventures in an Embedded World for Embedded.com .