What Is Decision Analysis?
Decision analysis workshops involve a group of decision makers with a facilitator that work within s structured decision model to analyze the merits of different options and determine which one(s) offer the best value. Decision analysis workshops can be used, but not limited, to:
- Evaluate project delivery options
- Prioritize actions
- Evaluate design options to be considered in a feasibility study
- Evaluate strategic investments
- Evaluate project design approaches
- Select vendors
- Select contractor that offers the “Best Value”
Special tools and technology are often applied to support the decision-making process.
The Value Metrics Decision Making Process
The Value Analysis Program VMS uses for Caltrans is a specific decision model, known Value Metrics, to support complex group decision making. Value Metrics is a decision-making process that leverages a powerful multi-attribute utility theory (MAUT) known as the Analytic Hierarchy Process (AHP). Stated simply, AHP breaks down complex decisions that include varied and disparate attributes into a series of smaller, pairwise comparisons utilizing a common ratio scale. From this structure, straightforward mathematical priorities may be derived that reflect relative degrees of preference for a set of alternatives. AHP was developed by Dr. Thomas Saaty at the Wharton School of Business in the 1970’s as widely accepted in both the business and academic worlds as one of the most sound approaches to structured decision making.
Value Metrics applies AHP, as well as elements of its sister techniques, the Analytic Network Process (ANP), in conjunction with a proprietary software system that utilizes remote control keypads to capture stakeholder preferences and priorities. The software synthesizes the subjective judgments of multiple stakeholders into explicit expressions of strength of preference that facilitates group decision making through shared learning and understanding. This approach leads to better group decisions by reducing the effect of cognitive biases.
In making value comparisons, four essential elements must be factored. These include cost, performance, time and risk. The Value Metrics approach to value measurement relies upon a fundamental mathematical algorithm for modeling value.
The dynamic model considers value as the interrelationship between inputs and outputs (or, alternatively, the costs and benefits) of a system. This type of model is non-linear in nature and considers tradeoffs between cost, time, and performance while considering how risk influences each of these elements. The outcomes realized are dependent upon the resources input into the system. This is why it can be said that it is a “dynamic” model of value. The algorithm for this model can be stated as follows:
Where V = Value, f = Function, P = Performance, C = Cost, t = Time, α = Uncertainty, and N = the number of developed value alternatives.
Value Metrics provides a standardized means of identifying, defining, evaluating, and measuring performance. It can be applied to all types of complex decisions including options analysis and selecting the best value proposal or vendor where quantitative and qualitative criteria are mixed together.
Value Metrics can improve group decision making by:
- Building consensus among project stakeholders (especially those holding conflicting views)
- Better informing decision makers regarding differing perspectives
- Making subjective judgments, and their strength of conviction, explicit
- Reducing bias that leads to suboptimal decisions
- Developing a better understanding of a decision’s goals and objectives and identifying and aligning decision criteria to them that will result in the desired outcomes
- Developing a deeper understanding of the relationship between performance, cost, time and risk in determining value
- Using value as the basis for making decisions
Provided below is an example of performance and value modeling of a decision involving the selection of “the best” option is performed using the Value Metrics approach.
In this example, a major transportation project in Southern California was evaluated by a large group of individuals representing 8 different federal, state and local agencies, all of which had different priorities and objectives. Using the Value Metrics process, the attributes were broken down into pairwise comparisons. Each stakeholder group was given a handheld remote control unit to input their preferences and priorities. The results are synthesized in the chart below, which represents the groups’ aggregate priorities for project performance. It is important to note that during this process, much debate and dialogue ensued, resulting in a much greater shared level of understanding of the individual perspectives, issues, and concerns held by the participants.
In the next chart, the same group rated the performance of two design options based on the performance attributes identified above. The individual performance ratings for each attribute are factored with the mathematical priorities established in the chart above to derive aggregate performance scores.
Finally, the group factors in the elements of cost and time to derive a total value comparison as illustrated in the chart below. VMS’ proprietary software does all of the heavy lifting – the group merely provides the data that informs the model. “What if” scenarios can be performed to test the sensitivity of decisions to the various decision criteria. All things considered, in this example, Alignment Option 2 offers the best value based on the preferences and priorities of the group.
Typical Decision Analysis Workshop Process
A VMS Caltrans Decision Analysis effort typically includes the following activities and can usually be addressed in a single workshop:
- Identify Project Sponsor(s)
- Identify the decision makers
- Identify the decision objective
- Identify the decision options
- Identify the decision criteria
- Determine the decision model
- Gather and data related to the options
- Develop measurement scales for decision criteria
- Prioritize the importance of the criteria
- Evaluate the decision options
- Review results
- Perform sensitivity analysis
- Develop a Decision Analysis Report