The Victorian Auditor General has released his report Realising the Benefits of Smart Meters. As with any program of this scale, there are many stakeholders, with many objectives. There were a series of assumptions that sit behind the decisions. In hindsight some of these assumptions have proven to be incorrect.

I, like you, have made decisions based on assumptions that I believed were right at the time. Looking back, some decisions could have been better had I challenged the assumptions more. I try to avoid commenting on decisions made if I was not involved in the discussions. It is easy to criticize when you don’t know the facts behind why someone made certain assumptions.

I will leave such commentary to the main stream media.  The likes of ABC News and Australian Financial Review are sharing their views on this report. Technology media such as IT News and ZDNet are also giving their interpretation of the report.

The report summary says, “Key expected benefits of smart meters were to:

  • improve consumers’ ability to monitor and control their electricity use, potentially allowing for cheaper and more efficient energy use
  • reduce the cost to industry of planning and managing power supply, potentially leading to lower retail prices for consumers
  • increase retail competition through new services, potentially resulting in a greater choice of retail offerings to consumers.”

Later in the report it states findings, such as:

  • benefits associated with the uptake of innovative tariffs and demand management―which has achieved only 2.5 per cent of expected benefits to be realised by 2014 
  • benefits that come from network operational efficiencies―which have achieved 49.32 per cent of expected benefits to be realised by 2014. 
  • early 2014 found that two-thirds of Victorians did not understand what the benefits of smart meters were and many were still unaware of the link between their smart meter and saving money on their electricity bills

This is a ~90-page report with many finding on cost and benefits. I have focused on what I believe to be the most important lessons relating to benefit realization. These lessons, expressed as questions below, should be asked during the business case phase. In fact they are standard questions you should ask during any business case phase:

  1. How will the market structure impact my ability to realise the stated benefits?
  2. Have we appointed benefit owners and do they agree with the underlying assumptions?
  3. Is it clear how we will realise these benefits?
  4. Do the assumptions have credible data to support them?

You won’t ever have 100% certainty to these questions. You need to manage the uncertainty as you would any other risk.

How will the market structure impact my ability to realise the stated benefits?

As the name states, Advanced Metering Infrastructure is an infrastructure investment. Costs in one part of the electricity value chain will deliver benefits in another.

Demand side management is a key benefit area for the Victorian business case. So let’s look at how you deliver demand side benefits. Just making information available to consumers will not result in large scale behaviour change. You need to get the consumers attention. You need to get them to change behaviour at a time you want them to. Usually this means you need to provide financial incentives.

In most parts of the world the retailer performs these activities. Delivering these outcomes requires investment from the retailer. They need to fund consumer outreach and education programs as well as offer incentives. The retailer may get rewarded by retaining the customer. They may also get indirect benefits via cross selling opportunities. A major benefit of reducing peak demand is deferring infrastructure investments.  For example, in generation, transmission and distribution. Another benefit is improving network performance through an increased load factor.

The Australian electricity market is deregulated. This means you rely on many actors collaborating to deliver an outcome. The two major benefits, deferral of asset investment and improvement of network efficiency would not be realised by the retailer. They would be realised by other actors in the value chain.

This is just one example of how market complexity may impact benefits realisation. Other examples in this report include the moratorium on retail pricing to customers. The complex mechanism to reflect operational cost savings in distribution businesses to  customer tariffs.  The national move to competitive metering.  All inject uncertainty and you need to treat as material risks.

Have we appointed benefit owners and do they agree with the underlying assumptions?

With any business case you must appoint benefit owners. Those owners must accept their role and the underlying assumptions made. In many business case processes this is difficult and often results in a top down decision. Regardless how a benefit owner gets appointed, it needs to happen. That owner needs to treat assumptions as risks as they are unknowns.

Looking at Victoria, who should have been accountable for delivering reduction in peak demand? The government, the retailer, the distributor?

Is it clear how we will realise these benefits?

Effective consumer outreach and education is critical to deliver demand side benefits. In Victoria you have  five distributors and twenty retailers. Each retailer would have a critical role to play as they own the relationship with the customer. Does the retailer have the right contact details for each customer, such as mobile phone number and email? Are those customers engaged via digital channels? Is that retailer analysing the load profile data to know who to target for demand side events? Are their customer information systems ready to offer such products? Is this a priority for that retailer to deliver these products and services? The list goes on and each unanswered question is a risk that you must mitigate. In parallel the distributor must ensure the timely collection of smart meter data happens. This is a complex ecosystem that all needs to work together to deliver the benefits.

Do the assumptions have credible data to support them?

Finally, you need to challenge the assumptions in the business case. It is a natural human behaviour to be optimistic, so assumptions tend to reflect this. Make sure you question the likelihood that an assumption will turn out to be accurate. Ask to see the data that support these assumptions.

As an example, the report says, “Collectively, these assumptions rely on 75 per cent of Victorian residential customers changing their consumption behaviour in response to incentives delivered by the AMI rollout“. What was the data that supported such an assumption? Most studies suggest consumers are not engaged when it comes to electricity behaviour change.

In conclusion

I support Advanced Metering as a strategic investment. In my experience it can deliver a positive return. It can position actors across the value chain to deliver many new and improved services. In most cases realising those benefits rely on effective community and consumer engagement.

Making a case in Victoria would have been complex given all the actors involved. Unless these actors collaborate you will not realise the benefits. Aside from industry actors, key actors are community groups and the consumer.

You must have clarity of ownership for all benefits. This includes those that rely on consumer behaviour change. You must have a plan for how these changes in consumer behaviour will result in actual benefits.

Ultimately, you must be comfortable assumptions that feed decision making are realistic.