The PV industry reacted with disappointment as the latest monthly estimates of the deployment of solar were published by BEIS a couple of days ago. Solar Power Portal said the numbers ‘revealed August to be the slowest deployment month yet under the new regime’.

There’s a big problem here. The statisticians at BEIS and its regulator, the UK Statistics Authority (UKSA), know that the numbers Solar Power Portal is quoting are systematically wrong. Both government bodies understand very well that the official figures are constructed in a such way that they will never accurately report the actual level of solar installation in any one month. For the last year I have working pro bono with the department and its regulator to try to get some improvement – or at least an open admission of the issues – in the way these numbers are presented. We've seen some progress - including far greater openness about the way retrospective revisions are made to the data - but serious unresolved problems persist.

This week I admitted failure in my attempt to get these issues addressed. For several months, BEIS and UKSA have been promising a final meeting at which the crucial amendments to the way the numbers are presented would be discussed. At first I was told the session would be held in September, then it was fixed for October 3, which was moved to the 27th, and then other dates were offered and finally on Monday I was informed that the earliest possible date at which the officials would be free was mid-November. That takes it up to almost a year since I first wrote to UKSA to make a formal complaint and I’ve now given up. BEIS seems unable to publicly acknowledge that many of its statistical practices – across the solar deployment statistics and other series - are seriously and deliberately misleading.

I’m writing this piece with sadness. I grew up with National Statistics, including a period briefly teaching economic statistics at university, and trusted government to produce honest and reliable figures because its statisticians are meant to be independent. The BEIS series which I have tried to help improve over the past year carry the National Statistics guarantee of quality. But despite the obvious flaws being now recognized by the department and the regulator, the reports are still allowed to carry this imprimatur today.

They shouldn’t be, and the failure of UKSA to insist on the quality mark being removed, makes me very concerned about all government data. If important figures are being produced that the statisticians know are wrong but they refuse to acknowledge the problems for fear of public or political criticism, we cannot be sure about any numbers coming out of government. This has serious implications for policy-making and for public trust.

The issues are complex, and I stress that I completely understand why BEIS finds it difficult to collect and present the data accurately. However its failure to acknowledge the problems is destructive to the PV industry and is misleading investors, the electricity market and policy makers. BEIS needs to openly admit the problem and, if necessary, stop producing the monthly document.

At heart, the cause of the statistical failure is this: PV installation data is poor and incomplete and arrives at BEIS very late. But the statisticians are obliged to bring out the figures every month. So what they do – understandably – is each month report what they know for certain has been installed in the previous month. But during the period since they last reported, previous month’s figures have been increased by data dribbling in about installations done before the previous month.

The effect is to make it seem as though each individual month has very low installation levels when they are first reported. But as time passes, the actual level is shown to be much larger, sometimes very much higher.

This year’s data is shown below. I find this information immensely confusing but I hope you understand it. The column is the month to which the data refers. The rows are the date of the report. So, for example, in the report for January (row) the amount of installed capacity for January (column) is recorded as 8,949 megawatts. In February, as more information has come in, the estimate of January capacity has risen to 9,202 megawatts. And the rise continues. By August, January’s total installed PV was estimate at 9,774 megawatts, over 800 megawatts higher than it initially said it was.

Why does the consistent rise – every month for every month - cause a problem? Because it obliges BEIS to underestimate the rate of monthly deployment. For the February report, it has only received confirmation of 11 megawatts of deployment in that month and it publishes that figure. So its statistical summary indicates it was a very poor month for PV. But look down at the bottom of the table; this shows that by August February is shown to be 87 megawatts higher than January.

This is closer to the real figure. But it is probably still an underestimate because BEIS continuously revises the numbers upwards for several years. It is still increasing its figures for PV from the months of 2014.

This point is even more apparent when we look at the figures for March. The first estimate of installations was 196 megawatts (9519 megawatts less 9323 megawatts). By August the estimate has risen five fold for the deployment in the month to 1018 megawatts.

Another way of looking it this is to compare how much PV BEIS has said in total has been installed each month with the figure for the absolute rise in PV now indicated. Add the together each month’s estimate of new deployment and you’d think the UK had put 376 megawatts of solar down so far this year. Then compute the difference between what BEIS said was the installed level in January (8,949 megawatts) and what it says now (11,034). This suggests a rise of at least five times as much. In time, the increase will be recorded as even greater because a lot of August data hasn’t arrived yet.

The underlying reason for this mess is that BEIS does not have access to decent data. But this doesn’t excuse the failure to acknowledge the problem. Last year, I initially complained to UKSA about the lack of any acknowledgement of the revisions. In fact, they were actively disguised by completely removing the data from public view.

As a result of my letters, UKSA forced BEIS to amend its policies on this issue and in several other important respects. The presentation of the data has become more transparent and honest. Nevertheless, the central problem remains. BEIS is presenting data each month under the National Statistics quality mark that is knows with absolute, unqualified certainty is wrong, and sufficiently wrong that it might affect both policy making and government expenditure by a noticeable amount.

This isn’t an isolated incident. A month ago, I complained that another series of BEIS data was in breach of National Statistics rules. Old statistical series were being wiped from the record to stop nosy people like me examining how BEIS had retrospectively changed its figures. Although I been initially told by a BEIS statistician that all the data had been ‘overwritten’, and therefore expunged, the Department did put back the old numbers on its web site two hours after I had submitted another formal complaint. You won’t be surprised to know that, once again, retrospective changes aren’t marked and anybody looking at the data will not be able to compute underlying growth and decline rates.

I think all this is very serious indeed. But I spend my life working with numbers and I’m probably not being objective.

Addendum,

Wookey (see the comments) asks for a chart that shows how one month's estimates are revised each month.

This is the chart he wants for January 2016. The numbers have risen by over 800 MW (0.8GW) since the first statistical report. This increment is, for example, enough to provide 3% of UK electricity demand on a sunny weekend day in June. The revision is hugely significant.

The X axis is the month of the revised report. So, for example, the deployment report for May says that January's total capacity was about 9,500 MW, up by about 100 MW from the previous month's estimate.