Summary

Specify measures to address measurement objectives.

Description

Measurement objectives are refined into precise, quantifiable measures.

Measurement of work can typically be traced to one or more measurement information categories. These categories include the following: service continuity, capacity, availability, service performance, and service quality.

Measures can be either base or derived. Data for base measures are obtained by direct measurement. Data for derived measures come from other data, typically by combining two or more base measures.

 

Examples of commonly used base measures include the following:
  • Estimates and actual measures of work product size (e.g., number of pages)
  • Estimates and actual measures of effort and cost (e.g., number of person hours)
  • Quality measures (e.g., number of defects by severity)
  • Information security measures (e.g., number of system vulnerabilities identified)
  • Customer satisfaction survey scores


 

Examples of commonly used derived measures include the following:
  • Earned value
  • Schedule performance index
  • Defect density
  • Peer review coverage
  • Test or verification coverage
  • Reliability measures (e.g., mean time to failure)
  • Quality measures (e.g., number of defects by severity/total number of defects)
  • Information security measures (e.g., percentage of system vulnerabilities mitigated)
  • Customer satisfaction trends


Derived measures typically are expressed as ratios, composite indices, or other aggregate summary measures. They are often more quantitatively reliable and meaningfully interpretable than the base measures used to generate them.

There are direct relationships among information needs, measurement objectives, measurement categories, base measures, and derived measures. This direct relationship is depicted for service work using some common examples in Table MA.1.



Example Work, Organizational, or Business Objectives Information Need Measurement Objective Measurement Information Categories Example Base Measures Example Derived Measures
Provide agreed service continuity Can services be recovered from disasters or major disruptions within agreed timeframes? Provide insight into whether the service continuity plans will be executed successfully to provide agreed service continuity Service continuity Number of services with recovery test failures
Total number of services in the service catalogue		 Service continuity confidence rate
Provide appropriate capacity to meet business need
Prevent capacity related incidents		 Are there enough resources (or too many) to meet demand for services? Provide insight into resource utilization, idle resources, and inadequate capacity to meet demand Capacity Total number of service requests
Available service provider staff hours
Service time		 Average service time
Service provider staff utilization
Provide cost effective service Is a cost-effective service being demonstrated through accurate capacity planning? Provide insight into unplanned capacity expenses Capacity Total expenses for unplanned capacity
Total costs for resources		 Percentage of service resource costs that are unplanned capacity expenses
Improve the level of service quality Is the level of service quality improving? Provide insight into whether the quality of service being delivered is improving by understanding how many errors are repeat errors Service quality Total number of repeat errors
Total number of errors		 Error repeat rate
Provide effective services How effective is the service? Provide insight into what percentage of service requests are being reworked Service performance Number of service requests reworked
Total number of service requests		 Service rework rate
Provide appropriate, agreed service availability Is appropriate, agreed service availability being provided? Provide insight into the availability of the service Availability Agreed service time
Downtime		 Availability
Is the service as reliable as agreed? Provide insight into the reliability of the service Availability Available time (in hours)
Total downtime (in hours)
Number of breaks in service (normal service is interrupted)		 Reliability as mean time between failure (MTBF)


Example Work Products



  1. Specifications of base and derived measures


Subpractices



1. Identify candidate measures based on documented measurement objectives.

Measurement objectives are refined into measures. Identified candidate measures are categorized and specified by name and unit of measure.



2. Maintain traceability of measures to measurement objectives.

Interdependencies among candidate measures are identified to enable later data validation and candidate analyses in support of measurement objectives.



3. Identify existing measures that already address measurement objectives.

Specifications for measures may already exist, perhaps established for other purposes earlier or elsewhere in the organization.



4. Specify operational definitions for measures.

Operational definitions are stated in precise and unambiguous terms. They address two important criteria:

  • Communication: What has been measured, how was it measured, what are the units of measure, and what has been included or excluded?
  • Repeatability: Can the measurement be repeated, given the same definition, to get the same results?



5. Prioritize, review, and update measures.

Proposed specifications of measures are reviewed for their appropriateness with potential end users and other relevant stakeholders. Priorities are set or changed, and specifications of measures are updated as necessary.