Many embedded systems developers have used agile techniques or their related precursors such as XP and scrum for a decade and have found development both more enjoyable and productive.
Not so long ago I attended a lecture on embedded systems development processes . The presenter’s view was that the techniques of embedded development haven’t greatly changed in the last 50 years, and hence neither need the process. He walked us through what was essentially Royce’s modified waterfall model – lots of time perfecting the specification, a throwaway prototype, and then a single design-build-test cycle with modest overlap. Change and incomplete requirements were evils to be rejected rather than accommodated. Agile methods had no place.
What better motivation for this post! There are many embedded systems developers, my colleagues and I included, who have been making use of agile techniques or their related precursors such as XP (extreme programming) and scrum for a decade, and have found development both more enjoyable and productive for it. There is of course no silver bullet; no one process fits all. But by sharing our highs and lows experienced in the use of agile techniques, I hope those who haven’t given an agile approach much consideration will be encouraged to do so, and those who have will enjoy an alternative perspective.
The presenter did briefly present the agile manifesto, a single slide like so:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
But the key postscript was missing from the slide, and that’s equally important:
That is, while there is value in the items on the right, we value the items on the left more.
In other words, there is still very much a place for documents, processes, and plans, and adopting an agile mind-set does not mean ditching these. Far from it; such activities are important where we have to agree designs with hardware teams, manage external dependencies, work within a regulatory framework, or hand over our work to another team for them to support, maintain, or update. The manifesto’s insight is that these activities alone are not enough.
The first statement is all about us, the developers. Best in class process or tools count for little if a team aren’t motivated to do the best that they can, aren’t working well together, or don’t have the right mix of expertise. The agile principles suggest that the best teams communicate face-to-face, organize themselves, maintain a constant sustainable pace, and are trusted to get the job done. Embedded systems development teams are no different.
The second statement, valuing working software more than comprehensive documentation, does worry some in the embedded world. They fear a descent into cowboy coding with no design activity and no documentation. But the agile approach doesn’t advocate skipping design and producing no documentation. Rather we break the work up in to a series of smaller deliveries, with each incremental delivery having its own design activity. Deliverables include documents as well as working software. We don’t try to deliver comprehensive documentation up front, only what is needed for the immediate small delivery. Techniques such as using user stories to capture requirements help us avoid getting too detailed too early.
Ideally the working software from each small delivery is actually useful to the customer; they can start deriving value from the development before all features are in place. Such early releases also enable them to see that progress is being made and reconfirm their view of what the product should do. But even if a release is not suitable for delivery to the customer, it enables us to validate design decisions and technology choices. By breaking projects into a series of shorter delivery cycles, we exercise the whole lifecycle so that we can start testing much earlier and get feedback on our work that benefits subsequent deliveries.
Does this mean the hardware team must break their work up in to a series of small deliverables aligned to software? If possible, yes, all of the above benefits apply equally well to hardware if they can work incrementally, by for instance adopting a modular design or providing functional but incomplete FPGA images. But where incremental hardware delivery is not an option this does not prohibit software working incrementally. Designing hardware abstraction layers allow the use of stubs and mocks to deliver software in advance of hardware.
There are many interesting techniques employed in the focus on working software, such as automated test, continuous integration, test-driven development, and code refactoring; we’ll visit all of these in later posts.
So on to the third statement. Many project failures are attributed to deliveries failing to provide the customer the value they had expected, due to a difference between what they thought they needed and what they actually needed. Similarly many errors in operation are traced to defects in the original requirements. Collaborating closely with a customer leads to a much higher likelihood that deliveries provide immediate value and avoids development of unnecessary features. The alternative; rigid adherence to a contract; inevitably leads to a flurry of change requests that delay releases. Costed change management may yield short term financial gain but is bad for long-term relationships and competitiveness. As embedded systems get ever more sophisticated, with rich user interfaces, configurable operation and wider interoperability the difficulty of capturing full requirements increases exponentially; maintaining an on-going conversation with the customer is crucial.
Of course collaborating with customers will result in many changes along a project’s course. Change comes from other quarters too, responding to competitors’ releases or accommodating complications and opportunities uncovered during implementation. The last manifesto statement encourages us to welcome change; by embracing it in a structured manner we can deliver value quicker. Dividing our work in to a number of smaller deliveries makes accommodating change much easier, if we haven’t started a feature yet then changing it is rarely costly, and if some development is complete the fact that we’ve shared this with the customer and got this feedback early minimizes the amount of rework required. This is equally true of documents; because we have held off designing a feature until the increment where it is delivered, document rework is similarly minimized when plans change.
My own experiences suggest that embedded development has changed. Our customers are highly aware of what can be achieved in software and expect sophisticated embedded systems. Our hardware is more powerful, has greater resources and integrates multiple interfaces and devices. This has increased the complexity of our solutions, the variety of technologies we use and the frequency that we roll out new features.
My hope with further posts on this blog is to demonstrate that adopting agile methods can be of great help to embedded developers. I’m interested to find out what your opinion is. What’s your experience of applying agile methods to embedded software development?