8.2.1 Purpose
The purpose of Analyze Performance Measures is to provide insights into the performance of a solution in relation to the value it brings.
8.2.2 Description
The measures collected in the task Measure Solution Performance (p. 166) often require interpretation and synthesis to derive meaning and to be actionable.
Performance measures themselves rarely trigger a decision about the value of a solution. In order to meaningfully analyze performance measures, business analysts require a thorough understanding of the potential value that stakeholders hope to achieve with the solution. To assist in the analysis, variables such as the goals and objectives of the enterprise, key performance indicators (KPIs), the level of risk of the solution, the risk tolerance of both stakeholders and the enterprise, and other stated targets are considered.
8.2.3 Inputs
- Potential Value: describes the value that may be realized by implementing the proposed future state. It can be used as a benchmark against which solution performance can be evaluated.
- Solution Performance Measures: measures and provides information on how well the solution is performing or potentially could perform.
8.2.4 Elements
.1 Solution Performance versus Desired Value
Business analysts examine the measures previously collected in order to assess their ability to help stakeholders understand the solution’s value. A solution might be high performing, such as an efficient online transaction processing system, but contributes lower value than expected (or compared to what it had contributed in the past).
On the other hand, a low performing but potentially valuable solution, such as a core process that is inefficient, can be enhanced to increase its performance level. If the measures are not sufficient to help stakeholders determine solution value, business analysts either collect more measurements or treat the lack of measures as a solution risk.
.2 Risks
Performance measures may uncover new risks to solution performance and to the enterprise. These risks are identified and managed like any other risks.
.3 Trends
When analyzing performance data, business analysts consider the time period when the data was collected to guard against anomalies and skewed trends. A large enough sample size over a sufficient time period will provide an accurate depiction of solution performance on which to make decisions and guard against false signals brought about by incomplete data. Any pronounced and repeated trends, such as a noticeable increase in errors at certain times or a change in process speed when volume is increased, are noted.
.4 Accuracy
The accuracy of performance measures is essential to the validity of their analysis. Business analysts test and analyze the data collected by the performance measures to ensure their accuracy. To be considered accurate and reliable, the results of performance measures should be reproducible and repeatable.
.5 Performance Variances
The difference between expected and actual performance represents a variance that is considered when analyzing solution performance. Root cause analysis may be necessary to determine the underlying causes of significant variances within a solution. Recommendations of how to improve performance and reduce any variances are made in the task Recommend Actions to Increase Solution Value (p. 182).
8.2.5 Guidelines and Tools
- Change Strategy: the change strategy that was used or is in use to implement the potential value.
- Future State Description: boundaries of the proposed new, modified, or removed components of the enterprise and the potential value expected from the future state.
- Risk Analysis Results: the overall level of risk and the planned approach to modifying the individual risks.
- Solution Scope: the solution boundaries to measure and evaluate.
8.2.6 Techniques
- Acceptance and Evaluation Criteria: used to define acceptable solution performance through acceptance criteria. The degree of variance from these criteria will guide the analysis of that performance.
- Benchmarking and Market Analysis: used to observe the results of other organizations employing similar solutions when assessing risks, trends, and variances.
- Data Mining: used to collect data regarding performance, trends, common issues, and variances from expected performance levels and understand patterns and meaning in that data.
- Interviews: used to determine expected value of a solution and its perceived performance from an individual or small group’s perspective.
- Metrics and Key Performance Indicators (KPIs): used to analyze solution performance, especially when judging how well a solution contributes to achieving goals.
- Observation: used to observe a solution in action if the data collected does not provide definitive conclusions.
- Risk Analysis and Management: used to identify, analyze, develop plans to modify the risks, and to manage the risks on an ongoing basis.
- Root Cause Analysis: used to determine the underlying cause of performance variance.
- Survey or Questionnaire: used to determine expected value of a solution and its perceived performance.
8.2.7 Stakeholders
- Domain Subject Matter Expert: can identify risks and provide insights into data for analyzing solution performance.
- Project Manager: within a project, responsible for overall risk management and may participate in risk analysis for new or changed solutions.
- Sponsor: can identify risks, provide insights into data and the potential value of a solution. They will make decisions about the significance of expected versus actual solution performance.
8.2.8 Outputs
- Solution Performance Analysis: results of the analysis of measurements collected and recommendations to solve performance gaps and leverage opportunities to improve value.
