The best architectures, requirements, and designs emerge from self-organizing teams. — Agile Principle #11
Of the 12 Principles behind the Agile Manifesto, this is the one with the smallest levels of adoption even among Agile Software Development teams, and the least well-understood. Most Agile experts and organizational development writers equate self-organization with autonomy — the ability of a team to make its own plans and decide for itself how to best accomplish its goals — and acknowledge that this is challenging to implement inside most corporate structures.
Some Agile methods, like Scrum, have adapted to the reality that team autonomy is inherently difficult. These methods try to preserve team ownership while accounting for the fact that these teams exist in a system that limits what they can organize independently. Others, like SAFe®, mention the importance of decentralized decision making but assume that teams receive strong direction from above.
For physical product teams, this principle may seem even more difficult because physical product development is the most cross-functional activity most organizations do. It touches not only the R&D engineers but also Marketing, Finance, Legal, Regulatory Affairs, Sourcing, Production, Quality, Sales and IT. Some of these groups have specialized contributions to make that are highly constrained by external requirements, needed at specific points in time, and may contribute to several programs at once. Others provide feedback early to drive effective implementation later, but don’t do much work themselves for most of the program. For all of these reasons, it’s hard for a product development team to have autonomy and build plans, without a strong project leader who knows how all the pieces fit together.
But all of these interpretations of self-organization focus only on one dimension of it — and the least important dimension if you seek to develop the best architectures, requirements and designs. For that, teams need to be put into the conditions that allow them to be truly self-organizing.
What Does It Mean to Be Self-Organizing?
The term “self-organization” comes out of chaos theory, where it has a specific meaning: the order that arises naturally in a disordered system under the right conditions. Crystallization is an example of this in the physical world: the disordered ions in saturated saltwater order themselves into regular crystals of sodium chloride as water evaporates. In 1986, two researchers applied that theory to product development and coined the term “self-organizing team.”
In 1986, Hirotaka Takeuchi and Ikujiro Nonaka wrote an influential paper in the Harvard Business Review called “The New New Product Development Game.” This paper is also the first place that “scrum” was applied to product development teams in the context of a rugby analogy. Jeff Sutherland, one of the authors of the Agile Manifesto, built Scrum from this paper and the discussions with the authors that it stimulated.
In that paper, Takeuchi and Nonaka make the case that the leading companies of the 1980s build speed and flexibility into product development by moving away from linear thinking and towards embracing the inherent uncertainty of doing something for the very first time. This is the realm of chaos theory where systems become self-organizing: as teams try to make decisions in the face of massive ambiguity, they develop their own dynamic order and their own independent agenda. Self-organization is not a principle the team uses, or permission the team has been granted.
Self-organization emerges as a natural property of a team operating at the boundaries of the possible.
Self-Organization Emerges When Teams Are Performing on the Edge
That paper stated that “A group possesses a self-organizing capability when it exhibits three conditions: autonomy, self-transcendence, and cross-fertilization.” Most thought leaders today focus only on autonomy: the team’s ability to determine its own work and set its own direction while management only sets a handful of constraints and provides the resources. If you focus only on that, then it’s easy to see why self-organization is so challenging, but not why it’s so difficult to sustain.
The two other properties describe why it’s so rare and so valuable. Self-transcendence isn’t even possible unless the team is working on something that is on the boundaries. The team transcends the stated goals of the program to push the limits, and then exceed them. Takeuchi and Nonaka describe how a Honda team tasked with developing a new car for the youth segment pushed beyond this original goal to produce a breakthrough product.
Cross-fertilization requires truly embracing cross-functional teams, starting in the earliest phases of development, so that all the perspectives get pulled together as the team builds out its ideas. All the examples cited in the paper were either pure hardware programs or they had both hardware and software (but 1980s software — not smartphone apps). They had cross-fertilization built into their team structures — and crucially, the teams did not hold themselves to rigid role definitions within these cross-functional teams in the earliest phases of their work. Even Agile software teams don’t include so many perspectives — they have a Product Owner who represents the user, and maybe a UX designer, but the rest of the team consists of software developers.
Most Product Development Teams Aren’t Self-Organizing but Need Autonomy
Few of the teams that come through my workshops are operating under the conditions that lead to self-organization. They are developing targeted improvements within product families where the technology is well-understood except for those areas being improved. They are much more highly constrained by time, cost, requirements and the need to work within established systems. They have cross-functional teams, but roles, expectations and deliverables are well-defined, even at the very beginning of development.
Such a team is not and will never be self-organized. But that doesn’t mean it’s stuck in command-and-control management either. Autonomy with respect to detailed plans and working methods has inherent value: it helps drive up motivation, builds team alignment and individual ownership, as it also leads to better plans that are more grounded in reality. An autonomous team — one able to organize its own daily work independent of management — is a team more likely to accomplish its objectives, even if everything else is highly constrained. They won’t be very innovative, but you don’t really need this type of product team to develop a breakthrough product when that’s not what they’re tasked with doing.
The Self-Organized Innovation Team
Teams operating at the boundaries of the possible — the teams developing your next big breakthrough — will be more effective if they are placed into conditions that drive self-organization. The inherent nature of that work has them living on the edge of uncertainty already. Management can help that by pulling together a cross-functional team, setting a goal that puts them far out on the edge of the possible, and then staying out of the way.
Nonaka described it this way: “Top management creates an element of tension in the project team by giving it great freedom to carry out a project of strategic importance to the company and by setting very challenging requirements.” They give the team time and money, a clear goal, and enough management attention to stay focused but not so much direction as to burden the project with an external agenda.
Teams like this will self-organize in the true sense of the word or fail trying — either outcome should be OK as long as the team and the organization learns from it. Under those conditions, they are poised to deliver the best architecture, requirements and designs.
Foster Autonomy Everywhere to Support Self-Organization Where Needed
All of this is easier to do if the organization already recognizes the value of autonomy. It will be easier to allow innovation teams to embrace the freedom they need for self-transcendence if your teams already have the ability to manage their own work and if middle managers already know how to build organizations of autonomous teams. The leaders will have built trust and attracted the best people to support ambitious innovation programs.
Finally, this is one principle that physical product teams need to embrace even more deeply than software teams, because self-transcendence requires cross-fertilization from people with such a wide variety of expertise. It is the only way to ensure that an innovative product can be built at scale to deliver sustained customer and business value.
To emphasize this, I’d rewrite the principle statement like this:
Breakthrough innovation arises out of teams that are given the conditions that allow self-organization to emerge. — Agile Principle #11
