“Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.” – Agile Principle #8

Software teams that develop products in a waterfall style can turn into “death marches” at the end — and hardware teams that operate under the same flawed model end up in “production hell.”

This means that the accumulation of missed requirements and design flaws have turned into a cascade of late-found defects that have overwhelmed the team. They are behind schedule, often months or even years late. The pressure to deliver is overwhelming — and it breaks people, even if they don’t talk about it openly.

You would think that the Age of Agile would have eliminated this among software teams but there are still managers who underestimate the effects of excessive stress on their teams — and ultimately their companies.

Teams find themselves working insane hours, sometimes sleeping at the office. Dinners consist of pizzas or whatever they grab from a fast food window or a vending machine. Exercise, sleep, time with their family and friends — all the things they do to stay healthy get squeezed out by the need to ship.

Long Hours Destroy Productivity

A research study from John Pencavel of Stanford University published in 2014 showed that working more than 50 hours per week causes a sharp drop-off in productivity. In 2005, Eric Robinson explained the economics of overload in “Why Crunch Time Doesn’t Work.” His work puts numbers behind the experience that sends teams out on death marches: after Hour 50 or so, the team has negative productivity.

The authors of the Agile Manifesto had witnessed both the personal destruction and the impact on their teams. When they began experimenting with the ideas that led to Agile, they hypothesized that a more continuous flow of development would help eliminate these overloads so that teams could work at a more sustainable pace.

Agile Software Development achieves this sustainability in three ways: the team does not take on more work in a cycle than they can accomplish in a normal work week, the team minimizes disruptions during the cycle and the team gets faster feedback to avoid a waterfall of defects at the end. These general principles maximize productivity across the board in product development.

“Production Hell” Is Unsustainable Development

Physical products experience the same unsustainable phenomenon in late development when manufacturability issues surface or early use tests show that customers are unhappy with some design decisions. This triggers a flood of rework that is now risky because of how integrated these decisions have become into the overall design, and expensive because the Last Responsible Moment is now in the past.

Engineers can fix one thing only to have three more problems crop up. They may have to travel to manufacturing sites or early customers who are far from home, with no idea when they’ll be able to come home. Leaders who are anxious about impact on business performance may demand frequent updates, so they have extra meetings on top of the demands of fixing the problem. The leaders often don’t like the decisions they have to make about cost vs. speed or about lower performance, so the meetings are filled with tension. The end result is a hellish experience for everyone.

What If You’re in Production Hell?

For a program that is in “Production Hell” one of the best things the team can do is to take a long weekend for sleep, exercise and reconnection with family. This will help recharge their batteries so that they make better decisions. It seems insane to take time off when the goal is so close, but that’s the best way to make sure that fatigue doesn’t lead to problems that push it further out.

For a leader with a team in this predicament, it’s better to back off on special status updates and mandatory work hours and ask the project leader how to get the information needed without disrupting the team. Sometimes the best thing to do is to leave them alone as much as possible and get updates from a single point of contact, usually the project leader or program manager, so that additional meetings don’t add to the stress.

Late Development Doesn’t Have to Be So Stressful

Even if the situation is not as difficult as described here, late development is often a lot more stressful than it has to be. Teams work overtime to fix production issues when “Crunch Time” is accepted as a normal part of the process. Products are often compromised in ways that were preventable, leading to disappointment and increased pressure on development teams. However, nothing a team does in late development can change this situation. The trajectory was set back in early development, and that’s where the problems have to be fixed.

In both hardware and software, the solution to problems in late development must be fixed in early development — by pulling learning forward before making decisions — and then validating decisions as soon as possible after they are made. In the software world, this is classic Agile Software Development: break the work down into small work packages and then make, execute and validate the decisions within those work packages in one iteration. This prevents almost all of the problems that send teams into death marches.

In fact, when we see that an “agile” team did get itself into trouble, we find that it’s usually because their managers have violated this principle: decisions were made without the knowledge needed to make good decisions. In the example I linked above, the developers didn’t have a clear understanding of some fundamental questions about the vision for the product and the vision kept changing. It’s difficult to make good design decisions if you don’t even know where you are supposed to be going.

This is especially true around the decisions about a product’s release date and what features would be included in the release. The principles of Agile suggest that teams can choose one or the other, but not both. As long as the team is working on the highest priority features first, and they have been working long enough to accurately predict their capacity, then it should be possible to project a release date that will include enough features to have a viable product in the market — or possible to put releases on a regular cadence and then release the features that are ready on the release date — without imposing Crunch Time.

Release Dates for Physical Products Come with Lead Times

We can’t easily adapt this model to physical products because we can’t just ship our products instantaneously. We have to build up production processes that come with lead time. We may be operating in consumer markets with rigid market windows, like the winter holidays, so the ship date can’t move. Or we may be in an environment where we have made commitments about dates to major customers.

This makes it even more important that we pull learning forward, because we cannot change course as quickly if we need to revisit earlier decisions, and the changes may be much more costly, which goes directly against the profitability of the product. It’s also important that we validate our decisions quickly through earlier feedback from manufacturing and better prototype testing, so that we can fix problems before they reach the expensive stages of development.

When we do that, there will still be late-found defects but there will be fewer of them. The team will not be as overwhelmed with the changes required. The major decisions — the ones the team flagged as Key Decisions — are much less likely to be revisited because the team consciously pulled learning forward for those decisions. Production may still feel like purgatory but with a clearer path to an end point.

This Principle Still Matters

We don’t need to make any changes to this principle to adapt it for physical products. We just need to recognize that this is important to our business, as well as to our people. And then we need to do what it takes to achieve it.

 “Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.” – Agile Principle #8