ESO teams up with Met Office on solar forecasting innovation project

How can greater accuracy in forecasting solar radiation help to balance the grid more effectively? An innovation project that brought together expertise from the Met Office and National Grid’s Electricity System Operator (ESO) energy forecasting team has achieved some significant results.

Debating the state of the weather is a great British tradition. Who doesn’t want to know if we can expect a sunny weekend ahead? But for some industries, including the energy sector, forecasting solar radiation is vital.

In 2016 National Grid ESO and the Met Office began a joint innovation project to enhance the accuracy of solar radiation forecasts provided by the Met Office to industry. These forecasts are generated by the Met Office's supercomputer which uses sophisticated forecasting models to process 215 billion observations gathered daily from around the globe, including around half a million from the Met Office's UK network of meteorological stations.

The project, which has now concluded, was funded by the Network Innovation Allowance (NIA), administered by Ofgem. The NIA programme supports innovation by electricity and gas network licensees to research and develop novel solutions to emerging problems that will ultimately benefit all energy consumers.

National Grid System Operator has a dedicated innovation team that sets the Innovation Strategy for the business across gas and electricity. This strategy sets out the innovation priorities for the System Operator, and therefore how innovation efforts and funding should be allocated. The innovation team works with the wider business to deliver that strategy by planning, funding and governing a portfolio of projects that tackle the System Operator’s innovation priorities.

Cian McLeavey-Reville, Innovation Strategy Manager at National Grid System Operator, explains: “Forecasting of supply and demand is one of our top innovation priorities and in this case we set out to achieve more accurate solar generation forecasting via enhanced solar radiation forecasts. This improved solar forecast will facilitate more efficient balancing of the network, which in turn would mean a lower overall cost to consumers.”

The challenge of predicting solar radiation

The Met Office uses a number of detailed models to provide ESO with solar forecasts. These range from within-day hourly forecasts up to 14 days ahead.

Senior Met Office Scientist Ian Pearman, who was part of the project team, explains: “Despite the massive computer power involved, it is still very challenging to predict exact solar radiation levels, particularly in longer range forecasts, because of the influence of clouds at local level. We all know how changeable Britain’s weather can be. Cloud cover can vary even over a distance of a few miles and on timescales of a few minutes. For example, sheets of cloud can quickly roll in from the North Sea turning sunny to overcast and then break-up and disperse just as abruptly.”

Why solar forecasting matters to the ESO

One of the ESO’s roles is to balance supply and demand in real time and as part of this work the Energy Forecasting team produces the national demand forecast which is essential to ensure the electricity system runs smoothly.

As more and more solar PV is connected, it is becoming an increasingly important part of the energy mix in terms of balancing supply and demand.

Yet, predicting the electricity produced by solar panels is inherently difficult. It is volatile, depends on the weather and most of the solar capacity is connected at distribution level, making it less visible to the ESO.

The route to more accurate forecasts

Solar forecast accuracy can be measured using an accepted metric known as ‘mean absolute error’, which measures the spread of the forecasts around the true value. The project team identified four packages of work to explore ways to reduce this error, as well as to reduce any bias.

“The first area we looked at was all about how we could develop models that improve our ability to forecast cloud because this has such a significant effect on solar radiation levels,” says Ian. “Another work package looked at our forecasting in the very short term (out to a few hours), which is significant to ESO’s operations. This is what we term our ‘Nowcast’.

“A third element of the study looked at our statistical processing – in other words, could we refine how we use the data from our observation stations to remove any biases.

“Finally, we studied the potential to introduce a weighted blended forecast taken from all our different models rather than the current approach that takes a simple average of the solutions from one ensemble forecast model.”

The results?

The switch to blended forecasts brought an improvement of between 5% and 10% in solar radiation forecast mean absolute error. The statistical processing efforts added a further improvement of between 1% and 2%. Ongoing investigations into cloud physics and implementing findings from the cloud forecasting work could bring even greater improvements.

So, does this advance in solar forecasting translate into more accurate demand forecasting by the ESO? Yes, according to Rob Rome, who heads-up the ESO’s Commercial operations team: “In order to meet our ambition to be able to operate a zero carbon electricity system by 2025, the ESO is committed to continuing to innovate to improve the accuracy of all national electricity demand forecasts.”

Kevin Tilley, Senior Energy Forecaster, who led the project for the ESO added, “As errors in weather forecasts can contribute up to half of our solar generation forecast error in certain conditions, this collaborative project with the Met Office was a natural one to dedicate our efforts to."

Kevin continued: “Over the last financial year, we’ve seen improvements of between 5 and 9% for the in-day and day-ahead national demand forecasts. One of the reasons for this improvement was the successful collaboration with the Met Office and the implementation of the project’s work into our forecasting operations.”

Wider benefits of the project

The improvements in solar forecasting will bring benefits beyond ESO, too. Other organisations in many different sectors rely heavily on knowing exactly when the sun will shine and for how long.

Ian Pearman explains: “These forecasting techniques use cutting-edge science and we believe our combined work on this project is already making a positive contribution to UK plc. For example, industries as diverse as aviation, transport and retail all use insight from weather forecasts to drive their decisions on everything from safe and efficient operations through to what stock to hold to meet consumer demand.

“The close working relationship we developed with the ESO team was a really important aspect to the project and it helped us focus on exactly where more accurate forecasting could make a difference.”