Measures

The final step in process improvement is to determine the impact on the organization of the changes that have been implemented.

This implies some set of measures which can be compared against a baseline in order to determine quantitatively how successful the process improvement program has been. To be effective, measurement should be integrated with an overall strategy for process improvement.

There are many approaches to defining measures and setting up a measurement program. MDM bases its work on the Practical Software and Systems Measurement project and the Goal-Question-Measures approach.

The Practical Software and Systems Measurement (PSM) project (psmsc.com) provides best practices used by measurement professionals. PSM is built around nine measurement principles:

Objectives and issues

are used to drive the measurement requirements. Project objectives are goals and requirements: cost, schedule, quality, functionality, and technical performance. Issues are areas of concern that present obstacles: problems, risks, and lack of information.

Define and collect measures

based on the technical and management processes. Measures should be collected as natural by-products of the work performed. Consider the processes of other team members and subcontractors as well as your own project processes.

Collect and analyze data

at a level of detail sufficient to identify and isolate problems. Periodically collect, process, and analyze measurement data. Specific data depends on project objectives and issues and the kinds of questions you need to answer.

Implement an independent analysis capability.

Measurement data should be assessed by an independent group. This ensures objectivity and accurate, unbiased appraisal of project status.

Use a systematic analysis process

to trace the measures to the decisions. The meaning of the numbers must be understood. There should be a clear flow from the data through the analysis to the conclusions. The analysis process should provide repeatable results.

Interpret the measurement results

in the context of other project information. No measurement result is good or bad by itself. A variance between planned and actual only indicates a possible problem, not the cause.

Integrate measurement into the project management process.

Measurement provides insight into the current phase. It also can project consequences of current actions on later phases.

Use the measurement process as a basis for objective communications

Involve the entire project in developing the measurement process. All parties should use same data and have a common understanding of the data definitions and commitment to the value of the measurement program.

Focus initially on project-level analysis.

Project success means meeting specific project objectives. Implement a consistent measurement process on all projects. Organization-level data can be derived from well-defined project measures.

RBWcommentsonnewwebsite.png

Goal-Questions-Measures (GQM)

The GQM approach is very useful in determining what measures to collect. The primary question is not "What metrics should I use?” but "What do I want to know or learn?" Because the answers depend on your goals, no fixed set of measures is universally appropriate. The goal-driven measurement process is based on three precepts:

Measurement goals are derived from business goals.
The goal-driven process begins with identifying business goals and breaking them down into manageable sub-goals.
Evolving mental models provide context and focus.
The primary mechanisms for translating goals into issues, questions, and measures are the mental models that you have for the processes you use. These mental models gain substance and evolve as you begin to make them explicit. They are the engines that generate the insights that guide you to useful measures and actions.
GQM translates informal goals into executable measurement structures.
The process ends with a plan for implementing well-defined measures and indicators that support the goals. Along the way, it maintains traceability back to the goals, so that those who collect and process measurement data do not lose sight of the objectives.

GQM involves these steps:

  • Identify your business goals.

  • Identify what you want to know or learn.

  • Identify your sub-goals.

  • Identify the entities and attributes.

  • Formalize your measurement goals.

  • Identify your measurement questions & indicators.

  • Identify the data elements.

  • Define and document measures and indicators.

  • Identify the actions needed to implement your measures.

  • Prepare a plan for implementing the measures.