“Working software is the primary measure of progress.” – Agile Principle #7
This is the principle from the Agile Manifesto that trips up hardware developers the most. Of course, hardware developers are not delivering working software — so what is the closest equivalent? And why does it matter?
Deliver Value in Small Batches
Development teams work best when they deliver their work in small batches, rather than all at once at the end of the program. They get test results faster, they can get user feedback faster, they can adapt to change with less disruption and they can fix problems that emerge faster.
The authors of the Agile Manifesto understood this. They had seen many software program teams deliver no value to the business until they delivered one large batch at the end of the program — with all the problems that go along with such a large release. The authors saw the potential to speed up the delivery of value in much smaller batch sizes, and they had already been experimenting with different ways to do that.
In fact, much of the variation between different “flavors” of Agile Software Development have to do with how small batches of work get defined and managed to optimize the flow of releases.
But that’s only true if the thing that’s delivered is “done-done” — if there are any loose ends, any unresolved defects, then these small iterative development cycles start to break down in the software world because teams can’t move on to the next User Story if the last one isn’t fully finished.
If the problems with work-in-progress software are especially challenging, then it’s difficult to know when the team will finish. Their estimates are likely to be wildly off. Most traditional project managers have stories of tasks that were 70% or 90% done for more than half of their final durations.
That led the authors to conclude that working software and ONLY working software gives an accurate assessment of the progress that a software development team has made. On the hardware side, this principle has caused the most grief of any, because it’s so difficult to translate.
Mistranslations in Many Agile for Hardware Approaches
The most common mistranslation requires hardware teams to act like software teams: break their work down into user stories or small tasks that can be driven to “done-done” in a single iterative cycle, and then “demo” their work at the end. This can cause teams to contort themselves into pretzels trying to make their work fit into boxes that just don’t make sense.
Many tasks in development can be broken down, but it doesn’t add a lot of value to do so — especially not if those things become items in a backlog that quickly grows unmanageable and over-constrained with resource and schedule dependencies. This doesn’t work well anywhere. We need to manage something bigger. In the software world, they’ve mostly organized around user stories as meaningful pieces of work.
A “user story” is a short definition of a feature from a user’s perspective. Mike Cohn of Mountain Goat Software provides a template: As a <type of user> I want <some goal> for <some reason.> That seems clear enough, and it’s easy to write statements like this for all types of products:
Pen: As a writer, I want to write smoothly and easily for more pleasurable journaling.
Tea Kettle: As a tea drinker, I want to boil water for making some nice tea.
Electric Vehicle: As a commuter, I want to recharge my car for driving to work tomorrow.
However, even in the simplest example of the pen, writing performance is an outcome of the whole system design. It requires attention to the ergonomics of the pen’s body, the interaction between the ink and the ballpoint or nib that delivers the ink, and how the pen moves across the paper. To fully fulfill this user story, we also have to consider the toxicity of the ink, the risk of leakage from the cartridge, the design of the cap so that the ink doesn’t dry out, and so on. It’s hard to deliver even the simplest user story without the whole system.
To make things worse, what is “done-done?” Is it when we have a single working prototype? Is it when we have the first successful production pilot run that fully validates our design? Or is it when the product is available on retail shelves? For this reason, almost nobody with any experience in hardware will base an Agile Hardware Development system on user stories in their pure form.
Misapplications of User Stories Dilute Their Power
Another common mistranslation is to redefine the “user story” to basically mean any small piece of work delivered from one component. In our Tea Kettle example, “As a safety protector, the temperature sensor wants to recognize boiling water so that it can send a signal that turns off the Tea Kettle without overheating.”
This is better, but still questionable in terms of value, especially because that’s still a high-level statement that needs more clarification to be actionable: does it turn off at 100°C, or does it somehow recognize boiling water at higher elevations by looking for temperature plateaus of a certain duration?
This is not the work of a single developer. The product engineer incorporates the heating element, thermostat and on/off switch into the design. They usually get help from Procurement or Supply Chain to source these parts from among suppliers that already make them. Then process engineers design the process for assembling the tea kettle from these and other components, and may require the assistance of software engineers to design automation routines for robotic steps in the process.
Then what does it mean to be “done-done?” When the sensor is in the CAD model and the BOM? When a sample has been tested at the subsystem level? When it’s built into a full system prototype? Or is it even later, after the process engineer is finished? To resolve these problems, some Agile experts stretch the definition of a user story even further, beyond all recognition until they are simply high level tasks.
All of this leads to the evolution of User Stories further and further away from their original intent, which means that the software teams also lose the rigor needed for User Stories to work well in a software context. At the end of the day, this approach doesn’t help anyone deliver more value.
The Value Delivered is Knowledge
For development teams, the ultimate value delivered is knowledge. In software, that knowledge is embodied in the code, so working code is the best measure of the value delivered. In other contexts, we need different definitions of value.
In the Rapid Learning Cycles framework, the value delivered is closed Knowledge Gaps, and the reports that capture the knowledge in a form that is easily consumed by decision makers and reused by later development teams. The primary purpose of a Learning Cycles Event is to share that knowledge, and then readjust the plan to accommodate feedback and new information.
Later in development, the value delivered could be the knowledge that the product’s design is working as expected, in the form of integration test results, or in the form of models and prototypes that build knowledge about the design. But the value is not the test, drawing, prototype or model itself. The value is the knowledge embodied in the test, drawing, prototype or model.
We don’t build prototypes or run tests because they are on the plan. We do these things to learn whether or not our designs are doing what we expect them to do, and sometimes to learn what we need to do in order to build a working system.
The Primary Measure of Progress Is the Knowledge
The authors of the Agile Manifesto worked in a world where knowledge was visible in the form of compiled software builds that passed specific tests and demonstrated specific functionality to the users: working software.
In the physical world, we have to update this principle to reflect how we make our own knowledge visible. We measure that by things like the number of high-priority Knowledge Gaps closed, the number of successful integration tests or the number of defects in a production prototype. All of these things ultimately measure the quantity and quality of the knowledge we have created that will enable our downstream partners to replicate, support and sell our products.
“The quality and quantity of knowledge created is the primary measure of progress.”
