Evaluation, Measurement, & Verification

Click here for a .pdf version of this page.

Work by ACEEE has consistently shown energy efficiency to be the least expensive, most quickly deployable, and cleanest of all energy resources.  As a recent report demonstrates the availability of good data is crucial to overcoming the significant barriers to implementing cost-effective energy efficiency.  Evaluation, Measurement & Verification (EM&V) methodologies can address this need by providing accurate, transparent and consistent metrics—based on good data—that assess the performance and implementation of an energy efficiency project or program.  These can then be tracked and compared both geographically and over time. 

Evaluation serves several purposes: accountability, risk management and improvement.  To restate these purposes as questions:

• Did the program deliver its estimated savings?  

  EM&V activities attempt to measure the effects of a program and determine whether it met its goals.  As resources pour into energy efficiency programs, good data on program impacts are needed to ensure that both ratepayer and taxpayer dollars are being well spent, and that programs are complying with any regulations. 

• How certain are these savings?  

  The issue of risk management is also a central concern.  Risk refers to the uncertainty surrounding the realization of expected savings from an efficiency project or program.  EM&V methodologies should be sophisticated enough to assess, and maximize, the level of confidence of estimated savings, thereby lending credibility to energy efficiency as a viable resource.  An added risk is that, in the absence of good data, governments may under-invest in relatively cheaper energy efficiency programs, and over-invest in more costly alternatives.  EM&V activities aim to provide this data, thereby avoiding costly misallocation of public and private resources.

• What can be done to improve program performance in the future?

  Most importantly, EM&V activities can—and should—be used to go beyond compliance by evaluating why a program had the effect that it did, with an eye to both improving existing programs and providing a robust mechanism for estimating savings from planned programs. 

Existing EM&V methodologies

It is important to make a distinction between energy efficiency projects and energy efficiency programs because of differences in the scope of measurement and methods of evaluation for each.  A project is a single activity that takes place at a single location, such as the installation of energy efficient lighting in an office.  In contrast, a program is a group of projects sharing similar characteristics and taking place in similar locations, such as a state-level effort to increase efficiency in all its buildings.

Evaluation methodologies for projects have existed for many years, the most widely used of which include the following:

• Federal Energy Management Program (FEMP) M&V Guidelines, Version 3.0 (2008)
• International Performance Measurement & Verification Protocol (IPMVP 2007)
• ASHRAE Guideline 14: Measurement of Energy and Demand Savings (2002)

At the program level, efforts by the National Action Plan for Energy Efficiency Leadership Group, co-facilitated by the DOE and EPA, led to the development of the Model Energy Efficiency Program Impact Evaluation Guide (2007).  This guide provides an in-depth discussion of EM&V program implementation, and is summarized below.

Steps in an EM&V Process

1. Define the evaluation objectives, scale, and time frame

   Evaluation planning should be included in the planning for the efficiency program itself, mainly for budgetary and staffing reasons.  The basic objectives of any evaluation program are accountability, risk management, and program improvement.  Other objectives may include the calculation of co-benefits, as described below.  Scale is often a tradeoff between expected benefit from the EM&V process and the administrative costs of the program.  Evaluation time frames are typically on the order of one year.

2. Select an evaluation method and define the baseline

   Evaluation methods depend on program objectives, and are discussed more fully in the referenced documents below.  The baseline (or "business-as-usual" scenario) consists of an estimate of energy use and demand in the absence of any efficiency program interventions.  Because energy savings cannot be directly measured, they must be calculated by comparing energy use and demand after efficiency program implementation with a baseline defined at the start of the program.

3. Calculate gross and net savings

   Gross savings represent the changes in energy use and demand that result from program activities, taking the influence of uncontrollable forces like weather into account.  A small sample of representative projects composing the program are selected, and their effects are measured and verified to determine gross savings.  Net savings are determined by accounting for externalities (free riders, spillover effects) from gross savings, and signify the energy savings that are directly attributable to the program.

4. Calculate co-benefits (according to policy objectives)

   Co-benefits include avoided greenhouse gas emissions and other environmental benefits, energy price effects, economic impacts such as job creation and increases in income, national security impacts, and other technical system benefits.  Methods exist for determining these co-benefits, according to the objectives of the energy efficiency program policy.