10.1.1 Purpose
Acceptance criteria are used to define the requirements, outcomes, or conditions that must be met in order for a solution to be considered acceptable to key stakeholders. Evaluation criteria are the measures used to assess a set of requirements in order to choose between multiple solutions.
10.1.2 Description
Acceptance and evaluation criteria define measures of value attributes to be used for assessing and comparing solutions and alternative designs. Measurable and testable criteria allow for the objective and consistent assessment of solutions and designs. The Acceptance and Evaluation Criteria technique can apply at all levels of a project, from high-level to a more detailed level.
Acceptance criteria describe the minimum set of requirements that must be met in order for a particular solution to be worth implementing. They may be used to determine if a solution or solution component can meet a requirement.
Acceptance criteria are typically used when only one possible solution is being evaluated, and are generally expressed as a pass or fail.
Evaluation criteria define a set of measurements which allow for ranking of solutions and alternative designs according to their value for stakeholders. Each evaluation criterion represents a continuous or discrete scale for measuring a specific solution attribute such as cost, performance, usability, and how well the functionality represents the stakeholders’ needs. Attributes that cannot be measured directly are evaluated using expert judgment or various scoring techniques.
Both evaluation and acceptance criteria may be defined with the same value attributes. When evaluating various solutions, the solutions with lower costs and better performance may be rated higher. When accepting a solution, the criteria are written using minimum performance requirements and maximum cost limits in contractual agreements and user acceptance tests.
10.1.3 Elements
.1 Value Attributes
Value attributes are the characteristics of a solution that determine or substantially influence its value for stakeholders. They represent a meaningful and agreed-upon decomposition of the value proposition into its constituent parts, which can be described as qualities that the solution should either possess or avoid.
Examples of value attributes include:
- ability to provide specific information,
- ability to perform or support specific operations,
- performance and responsiveness characteristics,
- applicability of the solution in specific situations and contexts,
- availability of specific features and capabilities, and
- usability, security, scalability, and reliability.
Basing acceptance and evaluation criteria on value attributes ensures that they are valid and relevant to stakeholder needs and should be considered when accepting and evaluating the solution. Business analysts ensure that the definition of all value attributes are agreed upon by all stakeholders. Business analysts may design tools and instructions for performing the assessment as well as for recording and processing its results.
.2 Assessment
In order to assess a solution against acceptance or evaluation criteria, it must be constructed in a measurable format.
Testability
Acceptance criteria are expressed in a testable form. This may require breaking requirements down into an atomic form so that test cases can be written to verify the solution against the criteria. Acceptance criteria are presented in the form of statements which can be verified as true or false. This is often achieved through user acceptance testing (UAT).
Measures
Evaluation criteria provide a way to determine if features provide the value necessary to satisfy stakeholder needs. The criteria are presented as parameters that can be measured against a continuous or discrete scale. The definition of each criterion allows the solution to be measured through various methods such as benchmarking or expert judgment. Defining evaluation criteria may involve designing tools and instructions for performing the assessment, as well as for recording and processing its results.
10.1.4 Usage Considerations
.1 Strengths
- Agile methodologies may require that all requirements be expressed in the form of testable acceptance criteria.
- Acceptance criteria are necessary when the requirements express contractual obligations.
- Acceptance criteria provide the ability to assess requirements based on agreed upon criteria.
- Evaluation criteria provide the ability to assess diverse needs based on agreed upon criteria, such as features, common indicators, local or global benchmarks, and agreed ratios.
- Evaluation criteria assist in the delivery of expected return on investment (ROI) or otherwise specified potential value.
- Evaluation criteria helps in defining priorities.
.2 Limitations
- Acceptance criteria may express contractual obligations and as such may be difficult to change for legal or political reasons.
- Achieving agreement on evaluation criteria for different needs among diverse stakeholders can be challenging.
