Copyright 2009 AACE, Inc.
AACE International Recommended Practices












AACE International Recommended Practice No. 42R-08

RISK ANALYSIS AND CONTINGENCY DETERMINATION
USING PARAMETRIC ESTIMATING
TCM Framework: 7.6 Risk Management
































Acknowledgments:
John K. Hollmann, PE CCE CEP (Author)
Rodney B. Adams, CCE
Hubertus M.T. Brandts, CCE
Alan J. Chilcott, CCT CCE
Dr. Ovidiu Cretu, PE
Charles J. Pospisil
Chinnadurai Ramachandran
Dr. Maarten S.A. Vrijland
Robert F. Wells, CEP


Copyright 2009 AACE International, Inc.
AACE International Recommended Practices
AACE International Recommended Practice No. 42R-08
RISK ANALYSIS AND CONTINGENCY DETERMINATION
USING PARAMETRIC ESTIMATING
TCM Framework: 7.6 Risk Management
January 26, 2009
INTRODUCTION

Scope

This recommended practice (RP) of AACE International (AACE) defines general practices and
considerations for risk analysis and estimating cost contingency using parametric methods. Parametric
methods are commonly associated with estimating cost based on design parameters (e.g., capacity,
weight, etc.); in this case, the method is used to estimate contingency based on risk parameters (e.g.
level of scope definition, process complexity, etc.). This RP includes practices for developing the
parametric methods and models (generally empirically-based). Recommended practice 43R-08 provides
example process industry parametric models (including software) [12]. For scheduling applications, there is
less research and reference material available; therefore schedule duration risk and contingency will be
covered in future revisions of the RP.


Purpose

This RP is intended to provide guidelines (i.e., not a standard) for contingency estimating that most
practitioners would consider to be good practices that can be relied on and that they would recommend
be considered for use where applicable. There is a range of useful contingency estimating
methodologies; this RP will help guide practitioners in developing or selecting appropriate methods for
their situation.

While this RP is relatively short, it incorporates a lot of information by reference and it addresses a
complex research and empirically based methodology. It is highly recommended that the reader
understands the research behind this method to avoid significant misunderstanding of risks and
misstatements of contingency.


Background

This RP is new. However, it is based on over 40 years of research, development, and practice. The
development and use of parametric risk analysis and contingency estimating methods evolved in parallel
with industry's recognition that poor project scope definition was often the greatest project cost and
schedule risk driver. This recognition led to the development of project scope development processes
(e.g., phase-gate processes) and scope definition maturity matrices such as those included in AACE's
recommended practice for cost estimate classification[1].

Before the above were accepted as best practices, experts first had to prove their value to project
outcomes. They did this by studying actual projects and developing empirically-based parametric models
that showed how poor scope definition resulted in greater cost growth and wider accuracy ranges. A
paper by Hollmann surveys these parametric developments[4] and highlights the pioneering work of the
late John Hackney, followed by Edward Merrow, et al. at the RAND Institute, and Steven Trost, et al. for
the Construction Industry Institute (CII)[5,6,7]. A paper by Baccarini also provides an extensive survey of
these methods[2]. These and the other sources referenced in this RP are recommended reading for
parametric method practitioners.

It is AACE's recommended practice that whenever the term "risk" is used, that the term's meaning be
clearly defined for the purposes at hand. In expected value practice as described in this RP, risk means
"an undesirable potential outcome and/or its probability of occurrence", i.e. "downside uncertainty (a.k.a.
threats)." Opportunity, on the other hand is "a desirable potential outcome and/or its probability of






Copyright 2009 AACE International, Inc.
AACE International Recommended Practices
Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009
2 of 8
occurrence", i.e. "upside uncertainty." The method in this RP quantifies the impact of uncertainty, i.e.
"risks + opportunities".


Background Parametric Estimating

This is not an RP on parametric estimating, but a basic understanding of it is required. AACE's Cost
Engineering Terminology defines a parametric estimate as one that has "estimating algorithms or cost
estimating relationships that are highly probabilistic in nature"[8]. Generally, the relationships of the
outcome (e.g., cost growth) and the inputs (e.g., risk drivers) are determined by studying empirical data
using methods such as multi-variable regression analysis, neural networks, or even trial and error. The
following illustrates the typical form of a simple parametric estimating algorithm:

Outcome = Constant + Coefficient 1*(Parameter A) + Coefficient 2*(Parameter B) +..



The "outcome" in this case may be a measure of cost growth (e.g., contingency percent), and the
parameters are various quantified risk drivers such as a measure of the level of scope definition upon
which the estimate or schedule was based. The algorithm can be much more complex employing
logarithmic, exponential, and power series.

Advantages of parametric estimating for risk analysis and contingency determination are that it is
inherently empirical in nature (based on actual measured experience) and it can directly provide
probabilistic information about the distribution of possible outcomes. It is also very quick and simple to
apply.

A disadvantage is that parametric estimating is based on empirical methods such as regression analysis
and these require that the parameters actually have more or less predictable relationships with the
outcomes. This is more important for some risk types than for others. Another disadvantage is that
obtaining empirical data and creating models is a challenging effort; increasingly so as one attempts to
model cost growth and risk drivers at more detail levels. Therefore, the method is typically limited in use
to estimating overall project contingency that results from select risk types. As will be explained in the
next section, this is not a problem for early estimates (i.e., AACE Class 5), but for later estimates (i.e.,
Class 4 or better) the method is best used in combination with range estimating, event tree analysis or
other more definitive methods.


Background Risk Types

In respect to parametric methods, risk types fall into one of two categories; risks that have systematically
predictable relationships to overall project cost growth outcome and those that don't. These categories
have been labeled as "systemic" and "project-specific" risks for contingency estimating purposes (i.e.,
there will be other ways to categorize risk types for other purposes.)[4]. In order to use the methods
properly, it is important to understand the distinctions of these types.

The term systemic implies that the risk is an artifact of the project "system", culture, business strategy,
process system complexity, technology, and so on. Research by Hackney and others has shown that the
impacts of some of these risks are measurable and predictable between projects within a system, and to
some extent within an industry as a whole. Measures of these risks are generally known even at the
earliest stages of project definition, and furthermore, the impacts of these risks tend to be highly dominant
for early estimates. Also, the link between systemic risks and cost impacts is stochastic in nature; this
means it is very difficult for individuals or teams to understand and to directly estimate the impact of these
risks on particular items or activities (for example, the risks of process technology on something like site
preparation or concrete foundations may be dramatic, but is not readily apparent). Finally, systemic risks
tend to be "owner" risks; i.e., the owner is responsible for early definition, planning, technology, and






Copyright 2009 AACE International, Inc.
AACE International Recommended Practices
Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009
3 of 8
decisions so these risks cannot be readily transferred to execution contractors. The following are typical
systemic risks dealt with using parametric methods:

Process Definition

Basic Design

Level of Technology

Process Complexity
Material Impurities
Project Definition

Site/Soils Requirements

Engineering and Design

Health, Safety, Security, Environmental

Planning and Schedule Development
Project Management and Estimating Process

Estimate Inclusiveness

Team Experience/Competency

Cost Information Available

Estimate Bias

One of the most difficult systemic risks to deal with is "estimate bias". When estimate bias is psychological
or political in nature, it is particularly difficult to measure and quantify because it deals with deception,
intentional or unintentional. To assess the impact of these types of risks (i.e., optimism bias and strategic
misrepresentation), a methodology called reference class forecasting (not covered here), a form of
estimate validation, has been proposed by Flyvbjerg[3]. Whether and how these systemic psychological
and political risks can be better measured, and incorporated in parametric techniques is an area of active
research, particularly for government funded (i.e., politically charged) infrastructure mega-projects. In any
case, estimate validation (to detect bias among other objectives) is always a recommended practice in
conjunction with risk analysis.

The term project-specific implies that the risk is, as it says, specific to the project. The impacts of these
risks are not highly predictable between projects within a system or within an industry as a whole. For
example, rain may have much more impact on one project than another depending on the project
characteristics and circumstances. Measures of these risks are generally not known at the earliest stages
of project definition (e.g., for Class 5 estimates, rain cannot be considered because the location of a
project, the season of its construction, and other circumstances may not be known). Also, the link
between project-specific risks and cost impacts is more deterministic in nature; i.e., they are amenable to
individual understanding and to estimating the impact of these risks on particular items or activities (for
example, the risks of excess rain on something like site preparation or concrete foundations can be
estimated). Finally, these types of risks are more negotiable during project contracting strategy as to who
will carry them. The following are typical project-specific risks (this list is far from inclusive):

Weather

Site Subsurface Conditions

Delivery Delays

Constructability

Resource Availability

Project Team Issues
Quality Issues (e.g., rework)

Etc.

This breakdown of risk types indicates why a combination of risk analysis and contingency estimating
methods should be used for optimal understanding and quantification of risks of different types. The RP
will explain how multiple contingency estimating methods can be used and their results combined. For
Class 5, parametric methods can be used alone given the knowledge of the systemic risk factors (and
lack of knowledge of project specifics) and the dominance of their impacts at this phase. Project-specific






Copyright 2009 AACE International, Inc.
AACE International Recommended Practices
Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009
4 of 8
risks become more dominant as scope is better defined (and hence some systemic risks are mitigated),
but there are always systemic risks that should be analyzed as thoroughly as practical.


RECOMMENDED PRACTICE

Practices for parametric risk analysis and contingency estimating methods necessarily focus on
development of the parametric model(s) because that is the most challenging aspect; use of parametric
models is relatively simple.


Model Development

Processes Come First
Prior to developing and using any risk analysis or contingency estimating practice, the enterprise's risk
management process should be developed in alignment with the appropriate overall asset management
and project control processes, which in turn should align with business strategy. Process maps show
inputs and outputs of a method which help identify stakeholders in its practice. Example processes are
covered in AACE's Total Cost Management (TCM) Framework[9]. In particular, if the company has no
formal project scope development process, or process or system for project historical database or
knowledge management, empirically-based parametric methods will be difficult to develop or maintain
(however, implementing parametric methods can put emphasis on the company's need to strengthen
these processes).

Determine Requirements
Company processes and stakeholder input will help establish requirements of the scope of the method
and scope of the effort to develop, maintain, and use it. Some typical requirements (and constraints) to
consider include:


Classes of estimates[1].: If your company is a contractor that only deals with Class 3 or better
estimates, and most systemic risks are carried by the owner, parametric methods offer less value.
However, for owner's developing Class 5 estimates, parametric methods are extremely valuable.


Types of projects and risks: If you estimate and fund projects using new technology, complex
processes, complex strategies, and so on, parametric methods increase in value and you will want to
be sure to identify and analyze these types of risks (in addition to the level of scope definition).


Corporate risk management strategies: If you are responsible for analyzing not only cost and
schedule risks, but also technical, health and safety or other kinds of risks, this may affect the
development process (this RP addresses cost risks)


Resources and competencies available: because of the reliance on empirical data analysis, the
development of models requires significant resources with special analysis skills (particularly
statistical). On the other hand, because the methods are very simple to apply, and because they
inherently incorporate empirical learnings, they can be used by project teams with less expert help
than other methods.

Historical Data
Having identified requirements in terms of the types of projects and risks to be addressed, the
requirements for historical or empirical data can be defined. The list of systemic risks provided previously
is a starting point; developers should study the references to this RP for more information on the specific
risk drivers to measure and capture. The primary risks are the level of scope definition, the level of new
technology in the process, and the complexity of the process and the project strategy. How to measure
and record these risks quantitatively must then be determined.







Copyright 2009 AACE International, Inc.
AACE International Recommended Practices
Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009
5 of 8
Having identified the risks (i.e., parameters of the model), measures of the outcome must be determined.
In general these include cost growth relative to the base estimate excluding contingency.

One systemic risk that is a challenge to measure is the competitiveness and quality of the base estimate.
"Fat" base estimates (i.e., hidden contingency above-the-line) may result in little need for or usage of
additional contingency. Therefore, a process to review and validate the competitiveness and quality of the
base estimate (and total including contingency) becomes an ancillary part of the risk management
process.

Having determined the parameters and outcomes to capture, data collection and management
procedures need to be established. Ideally, these will be part of your project historical database
management process, including project close-out processes.

Reference and External Information
As mentioned, the references to this RP should be studied. The Hackney and Merrow references include
models that have been developed from industry data, and are still generally applicable. AACE has
documented these models in RP 43R-08. These models can serve as a starting point or go-bys for
internal developments. Other external data on risks and their outcomes from benchmarking sources and
other literature [e.g., AACE's technical library and Professional Practice Guides (PPGs)] should be
obtained.

Data Analysis and Tool Development
Having collected project risk and outcome data including quantitative measures for modeling, it must be
cleaned to ensure that the sample to be used for model building or evaluation is free of significant error
and is representative (i.e., no extreme outliers that tend to bias analyses). Outcome data must also be
normalized for (i.e., corrected for) escalation, currency, and scope change impacts which are not covered
by contingency.

Two methods of parametric model building are commonly found in the literature. The most traditional and
widely used is multi-variable linear regression analysis. Standard spreadsheet software generally has this
analytical capability. The other more recent method is neural network analysis. This analytical
functionality is not included in standard "office" software such as spreadsheets and requires special
software designed for neural network analysis. The model building methods used for risk analysis and
contingency estimating tools are the same as those used for general estimating models; the only
difference is in the nature of the parameters and outputs.

Regression analysis will typically find some sort of relationship between one or more of the parameters
and the outcome measure. However, the relationship must be tested and challenged to ensure that it is
statistically significant (e.g., using t or F statistics), that it is causal in nature (i.e., there should be a
rational hypothesis for why a parameter is impacting the outcome to the extent noted), that the variables
are independent and not co-linear, and that the model is not overly biased by outlier data points, and so
on.

Once a valid model is obtained, it is usually implemented in a spreadsheet tool wherein the user enters
the parameter values and the model generates the predicted contingency value, usually as a percentage
of the base estimate value. The regression output represents the mean contingency which for normally
distributed data is equivalent to the p50 value (50 percent of the time the result will be over or under this
value).

After a base model is built, analysts can supplement the base model constants, coefficients, and
parameters with various logical assumptions and adjustments that may not have been included in the
analytical dataset[10]. For example, if database included a set of projects for which project definition was
rated on a scale of 5 to 1 using AACE's scope development maturity matrix (i.e., from RP 18R-97), and
later, AACE adds a new risk-driving deliverable to the maturity matrix, the analyst may have to make
manual adjustments to their model as appropriate to address how the change may affect the 5 to 1 rating.






Copyright 2009 AACE International, Inc.
AACE International Recommended Practices
Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009
6 of 8

Probabilistic Outcomes
The base model generates the mean or p50 result value. However, best practice for risk analysis and
contingency estimating is to produce a distribution of possible outcomes so that management can decide
how much risk they are willing to accept and therefore how much contingency will be required. The
regression analysis will provide some evidence of the probability distribution. In particular, it provides the
standard error of the estimate for the regression model dataset. However, the regression dataset may be
limited in scope, and cannot always be relied on to fully represent the range of possible outcomes.

There is a simple method, which is consistent with observed industry data (including AACE's RP 18R-97),
to generate a reasonably reliable probability distribution for cost contingency. That method is to assume
that cost outcomes (after allowing for contingency) are more-or-less normally distributed and to further
assume that contingency is equal to the standard deviation of the distribution[11]. With these assumptions,
the normal cumulative distribution can be computed using the NORMINV function in MS Excel [syntax is
NORMINV(probability, mean, std. dev)]. The following is an example of such a distribution.

Given:
Base Estimate (without contingency)

= $100
Contingency from the parametric model

= $20
Total Cost (at p50) = $100 + $20


= $120

Then the Cumulative Probability Distribution is:

p
Total$
NORMINV (probability,120,20)
Contingency$
(Total-Base)
10%
$ 94
$ (6)
20%
$ 103
$ 3
30%
$ 110
$ 10
40%
$ 115
$ 15
50%
$ 120
$ 20
60%
$ 125
$ 25
70%
$ 130
$ 30
80%
$ 137
$ 37
90%
$ 146
$ 46

These results can be reported in the tool in tables or charts as desired.


Dealing With a Lack of Company-Specific Historical Data

Unfortunately, good project data is difficult to collect and analyze. Fortunately, systemic risks and their
impacts for industry projects have been fairly consistent with time. Therefore, lacking any other method,
the parametric models from Hackney and Merrow can be used with some confidence after validating
against your own experience. The models have been included in working versions in recommended
practice 43R-08[12]..


Risk Analysis and Model Use

Identify and Quantify Systemic Risks
Because the parametric model has pre-determined risks (i.e., the parameters), the risk analysis is
simplified. While this is not an RP on how to conduct a risk analysis session or workshop, the typical
practice is to hold a meeting of the key team members and other project stakeholders, and to start with
identifying risks. In this case, the risk types are identified, so the team concentrates on quantifying the






Copyright 2009 AACE International, Inc.
AACE International Recommended Practices
Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009
7 of 8
parameters; e.g., rating the level of definition of each key deliverable in the project scope maturity matrix,
rating the level of new technology, and so on.

The more difficult challenges are agreeing on subjective systemic risk drivers such as the quality of the
base estimating data and the project team's competency. Because these types of risks are in fact
"systemic" (i.e., an artifact of the company's culture and capabilities that the project cannot do much
about), it is recommended that default ratings be set for these to avoid over-optimism. The ratings can be
changed, but the team must provide specific reasons why this project "bucks-the-system".

Estimating Contingency
Once the parameters are quantified, the contingency and probability distribution for systemic risks are
estimated by simply plugging the parameter values in the model. The user should make quality checks
and validate that the results are reasonable before reporting them to management.

Coordinate with Contingency Estimates for Project-Specific Risks
For Class 5 estimates, parametric methods alone are generally adequate, given the dominance of
systemic risk impacts and lack of knowledge of project specifics. For Class 4 or better, other methods
such as range estimating or event tree analysis should be used in combination with the parametric
analysis. These methods are covered in other RPs.

The most important consideration in combining methods and outcomes is to ensure that risks are not
double counted. After risks are identified in a risk analysis session, each risk must be categorized as
systemic or project-specific. Each risk is then quantified in their respective analyses and contingency
estimates.

Parametric and event tree analysis can be easily combined because event tree (i.e., expected value)
models work by directly estimating the probable cost distribution of the impacts of each risk. In that case,
the results of the parametric model (including its probability distribution) are included in the event tree
analysis as the first risk. Then other project-specific risks (e.g., heavy rain) are quantified and added to
the model. Monte-Carlo simulation can then be applied to the entire combined cost risk model to obtain a
combined probability distribution.

If range estimating is used for project-specific risk analysis, the combination cannot be done through a
combined Monte Carlo simulation to obtain an overall cost outcome distribution. This is because range
estimating does not model the cost impacts of each risk, but the cost range (resulting from many risks) of
critical items in the estimate. Another challenge is that range estimating recommends that the team
consider the extremes for the minimum and maximum cost of critical items and it is difficult, if not
impossible, to parse the impact of any particular risk. For these reasons, it is not the preferred
combination of methods. However, if care is take in not double counting the impact of system and project-
specific risks, the cost values at the various levels of probability can be added for these two methods.


Summary

It is hoped that enough information is provided in this RP to help guide practitioners in developing or
selecting appropriate methods for their situation. Users are encouraged to study the reference materials
provided with this RP. Future revisions of the RP are expected to cover scheduling applications.


REFERENCES

1. AACE International, Recommended Practice No. 18R-97, Cost Estimate Classification SystemAs
Applied in Engineering, Procurement, and Construction for the Process Industries, AACE
International, Morgantown, WV, (latest revision).






Copyright 2009 AACE International, Inc.
AACE International Recommended Practices
Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009
8 of 8
2. Baccarini, David, The Maturing Concept of Estimating Project Cost Contingency - A Review, 31st
Australasian University Building Educators Association Conference (AUBEA 2006): July, 2006
(http://espace.lis.curtin.edu.au/).
3. Flyvbjerg, B., From Nobel Prize to Project Management: Getting Risks Right, Project Management
Journal, August 2006.
4. Hollmann, John K., The Monte-Carlo Challenge: A Better Approach, AACE International
Transactions, AACE International, Morgantown, WV, 2007.
5. Hackney, John W. (Kenneth H. Humphreys, Editor), Control and Management of Capital Projects,
2nd Edition, AACE International, 1997.
6. Merrow, Edward W., Kenneth E. Phillips, and Christopher W. Meyers, Understanding Cost Growth
and Performance Shortfalls in Pioneer Process Plants, (R-2569-DOE), RAND Corporation, 1981.
7. Trost, Steven M. and Garold D. Oberlender, Predicting Accuracy of Early Cost Estimates Using
Factor Analysis and Multivariate Regression, Journal of Construction Engineering and Management,
Volume 129, Issue 2, pp. 198-204 (March/April 2003)
8. AACE International, Recommended Practice No. 10S-90, Cost Engineering Terminology, AACE
International, Morgantown, WV, (latest revision).
9. Hollmann, John K., Editor, Total Cost Management Framework: An Integrated Approach to Portfolio,
Program, and Project Management, (Chapter 7.6 Risk Management), AACE International,
Morgantown WV, 2006.
10. Whiteside, J.D., Contingency Modifier Tool, AACE International Transactions, AACE International,
Morgantown, WV, 2007.
11. Rothwell, Dr. Geoffrey, Cost Contingency as the Standard Deviation of the Cost Estimate, Cost
Engineering, AACE International, Morgantown, WV, July 2005.
12. AACE International Recommended Practice No. 43R-08, Risk Analysis and Contingency
Determination Using Parametric Estimating Example Models As Applied for the Process Industries,
AACE International, Morgantown, WV, (latest revision)


CONTRIBUTORS

John K. Hollmann, PE CCE CEP (Author)
Rodney B. Adams, CCE
Hubertus M.T. Brandts, CCE
Alan J. Chilcott, CCT CCE
Dr. Ovidiu Cretu, PE
Charles J. Pospisil
Chinnadurai Ramachandran
Dr. Maarten S.A. Vrijland
Robert F. Wells, CEP