Chapter 10 – 10.19 – Estimation

10.19.1 Purpose

Estimation is used by business analysts and other stakeholders to forecast the cost and effort involved in pursuing a course of action.

10.19.2 Description

Estimation is used to support decision making by predicting attributes such as:

• cost and effort to pursue a course of action,
• expected solution benefits,
• project cost,
• business performance,
• potential value anticipated from a solution, and
• costs of creating a solution,
• costs of operating a solution,
• potential risk impact.

The result of estimation is sometimes expressed as a single number. Representing the results of estimation as a range, with minimum and maximum values along with probability, may present a higher degree of effectiveness for stakeholders.

This range is referred to as a confidence interval and serves as a measure of the level of uncertainty. The less information that is available to the estimator, the wider the confidence interval will be.

Estimation is an iterative process. Estimates are reviewed as more information becomes available, and are also revised (if appropriate). Many estimation techniques rely on historical performance records from the organization in order to calibrate estimates against prior experience. Each estimate can include an assessment of its associated level of uncertainty.

10.19.3 Elements

.1 Methods

Various methods of estimation are used for specific situations. In each case it is important for the estimators to have an agreed-upon description of the elements to be estimated, often in the form of a work breakdown structure or some other decomposition of all the work being estimated. When developing and delivering an estimate, constraints and assumptions also need to be clearly communicated.

Common estimation methods include:

• Top-down: examining the components at a high level in a hierarchical breakdown.
• Bottom-up: using the lowest-level elements of a hierarchical breakdown to examine the work in detail and estimate the individual cost or effort, and then summing across all elements to provide an overall estimate.
• Parametric Estimation: use of a calibrated parametric model of the element attributes being estimated. It is important that the organization uses its own history to calibrate any parametric model, since the attribute values reflect the skills and abilities of both its staff and the processes used to do work.
• Rough Order of Magnitude (ROM): a high-level estimate, generally based on limited information, which may have a very wide confidence interval.
• Rolling Wave: repeated estimates throughout an initiative or project, providing detailed estimates for near-term activities (such as an iteration of the work) extrapolated for the remainder of the initiative or project.
• Delphi: uses a combination of expert judgment and history. There are several variations on this process, but they all include individual estimates, sharing the estimates with experts, and having several rounds of estimation until consensus is reached. An average of the three estimates is used.
• PERT: each component of the estimate is given three values: (1) Optimistic value, representing the best-case scenario, (2) Pessimistic value, representing the worst-case scenario, (3) Most Likely value. Then a PERT value for each estimated component is computed as a weighted average: (Optimistic + Pessimistic + (4 times Most Likely))/6.

.2 Accuracy of the Estimate

The accuracy of an estimate is a measure of uncertainty that evaluates how close an estimate is to the actual value measured later. It can be calculated as a ratio of the width of the confidence interval to its mean value and then expressed as a percentage. When there is little information, such as early in the development of a solution approach, a Rough Order of Magnitude (ROM) estimate is delivered, which is expected to have a wide range of possible values and a high level of uncertainty.

ROM estimates are often no more than +50% to -50% accurate. A definitive estimate, which is much more accurate, can be made as long as more real-world data is collected. Definitive estimates that are used for predicting timelines, final budgets, and resource needs should ideally be accurate within 10% or less. Teams can combine the use of ROM estimates and definitive estimates throughout a project or initiative using rolling wave estimates. A team creates a definitive estimate for the next iteration or phase (for which they have adequate information), while the remainder of the work is given a ROM estimate. As the end of the iteration or phase approaches, a definitive estimate is made for the work of the next iteration or phase and the ROM estimate for remaining activities is refined.

.3 Sources of Information

Estimators consider available information from prior experience along with the attributes being estimated.

Some common sources of information include:

• Analogous Situations: using an element (project, initiative, risk, or other) that is like the element being estimated.
• Organization History: previous experiences of the organization with similar work. This is most helpful if the prior work was done by the same or a similarly-skilled team and by using the same techniques.
• Expert Judgment: leveraging the knowledge of individuals about the element being estimated. Estimating often relies on the expertise of those who have performed the work in the past, internal or external to the organization. When using external experts, estimators take into account the relevant skills and abilities of those doing the work being estimated.

.4 Precision and Reliability of Estimates

When multiple estimates are made for a particular attribute, the precision of the resulting estimate is a measure of agreement between the estimates (how close they are to each other). By examining measures of imprecision such as variance or standard deviation, estimators can determine their level of agreement.

The reliability of an estimate (its repeatability) is reflected in the variation of estimates made by different methods of estimating or by different estimators.

To illustrate the level of reliability and precision, an estimate is often expressed as a range of values with an associated confidence level. That is, for a given summary estimate value and confidence level, the range of values is the expected range of possible values based on the estimates provided. For example, if a team estimated that some task would take 40 hours, a 90% confidence interval might be 36 to 44 hours, depending on what they gave as individual estimates. A 95% confidence interval might be 38 to 42 hours. In general, the higher the confidence level in the estimate, the narrower the range would be.

To provide estimates with a particular level of confidence, estimators can use a technique such as PERT. Using the multiple estimates for each component of the estimate, a probability distribution can be constructed. This distribution provides a way to compute an overall estimate (incorporating all of the estimated elements) as a range of values, with an associated level of confidence.

.5 Contributors to Estimates

The estimators of an element are frequently those responsible for that element. The estimate of a team is usually more accurate than the estimate of one individual, since it incorporates the expertise of all team members.

In some cases, an organization has a group that performs estimation for much of the work of the organization. This is done with care, so that the estimate reflects the likely context of the element being estimated.

When an organization needs a high level of confidence in the estimate of some critical element, it may call on an external expert to perform or review the estimate. The organization may compare an independent estimate against their internal estimate to determine what adjustments may be needed.

10.19.4 Usage Considerations

.1 Strengths

• Estimates provide a rationale for an assigned budget, time frame, or size of a set of elements.
• Without an estimate, teams making a change may be provided an unrealistic budget or schedule for their work.
• Having a small team of knowledgeable individuals provide an estimate by following a defined technique generally results in a closer predictor of the actual value than if an estimate was made by one individual.
• Updating an estimate throughout a work cycle, in which the estimated elements are refined over time, incorporates knowledge and helps ensure success.

.2 Limitations

• Estimates are only as accurate as the level of knowledge about the elements being estimated. Without organization or local knowledge, estimates can vary widely from the actual values determined later.
• Using just one estimation method may lead stakeholders to have unrealistic expectations.