Welcome to a series on Agile Systems Engineering, exploring the practical aspects of this emerging approach. If you didn’t see it already, check out Part 1: What is Agile, Anyway? and Part 2: What’s Your Problem?
The antithesis of agile
Requirements are a poor way to acquire a system. They’re great in theory, but frequently fail in practice. Writing good requirements is hard, much harder than you’d think if you’ve never had the opportunity. Ivy Hooks gives several examples of good and bad requirements in the paper “Writing Good Requirements“. Poor requirements can unnecessarily constrain the design, be interpreted incorrectly, and pose challenges for verification. Over-specification results in spending on capabilities that aren’t really needed while under-specification can result in a final product that doesn’t provide all of the required functions.
If writing one requirement is hard, try scaling it up to an entire complex system. Requirements-based acquisition rests on the assumption that specification and statement of work are complete, consistent, and effective. That requires a great deal of up-front work with limited opportunity to correct issues found later. A 2015 GAO report found that “DoD often does not perform sufficient up-front requirements analysis”, leading to “cost, schedule, and performance problems”.
And that’s just the practical issue. The systematic issue with requirements is that the process of analyzing and specifying requirements is time consuming. One of the more recent DoD acquisition buzzphrases is “speed of relevance”. Up-front requirements are antithetical to this goal. If it takes months or even years just to develop those requirements, the battlefield will have evolved before a contract can be issued. Add years of development and testing, and then we’re deploying last-generation technology geared to meeting a past need. That’s the speed of irrelevance.
Agile promises a better approach to deliver capabilities faster. But we have to move away from large up-front requirements efforts.
Traditional requirements-based acquisition represents a fixed scope, with up-front planning to estimate the time and cost required to accomplish that scope. Pivoting during the development effort (for example, as we learn more about what is required to accomplish the mission) requires re-planning with significant cost and schedule impacts. The Government Accountability Office (GAO) conducts annual reviews of Major Defense Acquisition Programs (MDAPs). The most recent report analyzing 85 MDAPs found that they have experienced over 54 percent total cost growth and 29 percent schedule growth, resulting in an average delay of more than 2 years.
Defense acquisition leaders talk about delivering essential capabilities faster and then continuing to add value with incremental deliveries, which is a foundational Agile and Dev*Ops concept. But you can’t do that effectively under a fixed-scope contract where the emphasis is on driving to that “complete” solution.
The opposite of a fixed-scope contract is a value stream or capacity of work model. Give the development teams broad objectives and let them get to work. Orient the process around incremental deliveries, prioritize the work that will provide the most value soonest, and start getting those capabilities to the field.
“But wait,” you say, “doesn’t the project have to end at some point?” That’s the best part of this model. The developer’s ‘fixed’ cost and schedule keeps getting renewed as long as they’re providing value. The contractor is incentivized to delivery quality products and to work with the customer to prioritize the backlog, or the customer may choose not to renew the contract. The customer has flexibility to adjust funding profiles over time, ramping up or down based on need and funding availability. If the work reaches a natural end point—any additional features wouldn’t be worth the cost or there is no longer a need for the product—the effort can be gracefully wrapped up.
You may be familiar with the project management triangle1. Traditional approaches try to fix all of the aspects, and very often fail. Agile approaches provide guardrails to manage all of the aspects but otherwise allow the effort to evolve organically.
The most important aspect of agile approaches is that they shift requirements development from an intensive up-front effort to an ongoing, collaborative effort. The graphic below illustrates the difference between traditional and agile approaches. With traditional approaches, the contractor is incentivized to meet the contractual requirements, whether or not the system actually delivers value to the using organization or is effective to the end user.
In an agile model, the development backlog will be seeded with high-level system objectives. Requirements are developed through collaboration among the stakeholders and the development is shaped by iterative user feedback. The agile contract may have a small set of system requirements or constraints. For example, it may be a requirement for the system to comply with an established architecture or interface, meet particular performance requirements, or adhere to relevant standards. The key is that the provided set of requirements are as minimal as possible.
The requirements discovery, analysis, and development process is collaborative, iterative, and ongoing. It really isn’t extremely different from a traditional requirements decomposition, as requirements still have to be traceable from top-level objectives. A key difference is that the decomposition happens closer to the development, both in time and organization. The rationale and mission context for a requirement won’t get lost because the development team is involved in the process, so they understand the drivers behind the features they’ll be implementing.
I’m getting ahead of myself, though! In the next installment of this series we’ll look at cross-functional development teams, the role of Product Owner, and scaling up to a large project.
What are your experiences with agile contracts and agile requirements? Share your best practices, horror stories, and pitfalls to avoid below.