The inaccuracy of cost estimates for developing major Department of Defense (DoD) systems is well documented, and cost overruns have been a common problem that continues to worsen. Because estimates are now prepared much earlier in the acquisition lifecycle, well before concrete technical information is available, they are subject to greater uncertainty than they have been in the past. Early lifecycle cost estimates are often based on a desired capability rather than a concrete solution. Faced with investment decisions based primarily on capability, several problems are encountered when creating estimates at this early stage:
- Limited Input Data –The required system performance, the maturity of the technology for the solution, and the capability of the vendors are not fully understood.
- Uncertainties in Analogy-Based Estimates – Most early estimates are based on analogies to existing products. While many factors may be similar, the execution of the program and the technology used as part of the system or to develop it are often different. For example, software product size depends heavily on the implementation technology, and the technology heavily influences development productivity. Size and productivity are key parameters for cost estimation.
- Challenges in Expert Judgment – Wide variation in judgment can exist between experts and the confidence in the input that they provide is generally not quantified and unknown.
- Unknown Technology Readiness – Technology readiness may not be well-understood, and is likely to be over or under estimated.
An Improved Method for Early Cost Estimation
In 2011 the SEI introduced the QUELCE (Quantifying Uncertainty in Early Cost Estimation) method, an integrative approach for pre-Milestone A cost estimation. The method aims to provide credible and accurate program cost estimates within clearly defined, statistically valid confidence intervals. QUELCE produces intuitive visual representations of the data that explicitly model influential relationships and interdependencies among the drivers on which the estimates depend. Assumptions and constraints underlying the estimates are well documented, which contributes to better management of cost, schedule, and adjustments to program scope as more is learned and conditions change. Documenting the basis of an estimate facilitates updating the estimate during program execution and helps others make informed judgments about estimation accuracy.
The QUELCE method differs from existing methods because it
- uses available information not normally employed for program cost estimation
- provides an explicit, quantified consideration of the uncertainty of the program change drivers
- enables calculation (and re-calculation) of the cost impacts caused by changes that may occur during the program lifecycle
- enhances decision-making through the transparency of the assumptions going into the cost estimate
Figure 1: Information Flow for Early Lifecycle Estimation, Including the QUELCE Method
Figure 1 shows the flow of information in a typical MDAP Acquisition, with blue boxes added to represent the contributions from the QUELCE method.