Phil Bishop
posted this
28 November 2018
- Last edited 13 February 2020
I'll take a look into this over the next day or two and will update this response if I discover anything of note. In particular, I'll check to see if what has been published on EMI datasets is identical to what we have in our data warehouse. But I'm going to predict now that my update won't change anything.
The Market Analytics team at the Authority inherited the Authority's data warehouse in mid-2013. The decision was made at that time to get the business (i.e. the users of the data such as ourselves) to take over the governance and management of the data we collect, store, use and publish from the IT team. One of the first things we set about doing was establishing a specification for all of the data we received. The so-called metered data had no specification; that remains the situation today. That said, we know where it comes from and what it is supposed to represent.
However, what was actually received and processed, and how it was processed, back in the years you mention might be difficult for us to do anything about now. I'd be shocked if we could even locate the source files. Moreover, we've been planning for quite some time now to replace the wholesale volume data that we publish with a series that we derive from reconciliation data sources going back to about 1 Jan 2005. Unfortunately, we haven't got to that task yet but hope to knock it off in 2019. When we do this, it will replace the published metered data series, although obviously not for the years you mention.
BTW, I assume you've seen this discussion on daylight saving.
Cheers
Phil
