While traditional “waterfall” processes (think Gantt charts) still have an important role in engineering program management – particularly for hardware programs and to plan anticipated development of large programs – time-box processes such as Agile Scrum that encourage short development cycles with frequent deliverables should be preferred. There are many very good reasons for this and there are many sources available to explain the merits of time-box methods. Her are a few:
- Large “monolithic” programs managed through the waterfall method allow for long development cycles that must each end before different pieces of the solution are pulled together into an overall product. As such, the “product” that is finally introduced into the test phase is very complex, with layers and layers of relatively untested software. Unexpected behaviors in the interworking of the different modules that have been developed over the course of many months is often very difficult to fathom and can create long schedule slips.
- Smaller time-box “sprints” to deliver incremental functionality limit the scope of any single software development iteration and result in fully tested software deliverables in short time horizons. Because functionality is completed and tested regularly, the likelihood of having major setbacks late in the game are dramatically reduced and schedules are much more predictable and linear.
- The best argument for time-box development, however, is the idea that you can have much more frequent delivery of new product value to your customers. While you may elect to combine the results of several time-box iterations before a given release, you have the choice to react to new market and customer demands in a much more responsive (Agile) way.
Some argue that Agile is too unpredictable because the methods support the idea that estimating the effort for a development activity cannot be done until the work is planned for a time box sprint. This is no more true of Agile methods than of any other, and serious Agile practitioners understand the need to estimate schedules and make business plans as well as anyone else. I also argue that waterfall methods only imply scheduling accuracy and – particularly for larger development undertakings – are rarely met. By using the time-box Agile methods, a product manager (Product Owner in Agile parlance) can more readily see how development is progressing, make salient decisions regarding feature content and better manage business results.
Waterfall planning has its uses, especially for hardware development or complex system level development. However, I favor using a combination of waterfall and Agile methods (stringing together time box iterations) to forecast large software or combined hardware and software programs. The sub-process or methodology for software execution should always be based on an Agile process, but I highly recommend using other tools to help manage it. Jira and Rally are two that I am familiar with, although there are many others.
I have already described the process to establish a roadmap. Mapping development items or individual cycles into engineering execution plans is relatively straightforward. From the prioritized list of programs, product management and engineering should create plans for the highest priority programs first. Prioritization should take everything that requires resources into account, including maintenance, customer requests and strategic roadmap items. Product Management should create market requirement documentation (or Agile user stories) and Engineering should respond to these requirements with resource estimates, implementation strategies and high level timelines for execution (requirements in the Gate Process described earlier). In keeping with Agile processes, planning should favor shorter development cycles, but also prioritize combining groups of complementing features or functionality that are important to particular sets of customers or market segments. In other words, each release should be sale-able and drive revenue if at all possible. As each of the highest priority plans is outlined and approved, the resources for that program must be set aside. This planning cycle continues until all resources are allocated, and starts again as programs are completed.
Whether you are using an Agile process or not, it is essential that the product management team work with the engineering leadership to always stay ahead of the engineering execution pipeline in the planning of what to do next. It can be very costly for large numbers of engineers to be idle while the leadership decides the details, strategy or architecture of what comes next. It requires significant discipline to stay ahead, but the best organizations will maintain this discipline to maximize NPI velocity and get the most bang for the buck out of engineering and product management budgets.
The Core Team operation discussed earlier is also critical to the efficient execution of the NPI process. In keeping with the gate process, members of the Core Team have significant roles to play in bringing new products to market. These activities should be tracked in their meetings and the team should be managed with an expectation of accountability. Functional leaders should be accountable for their team’s support of Core Team deliverables. For example, well ahead of the launch of a new program, marketing must prepare go-to-market plans and collateral, and operations must prepare manufacturing sourcing, build plans, quality plans, forecasting (with sales) and plans to ramp production. When operating with a sense of urgency and empowerment, Core Team execution streamlines NPI execution. But again, I emphasize that functional leadership must support Core Team execution, mentoring and coaching the Core Team members toward continuous improvement and best in class operational excellence.