Historical generation (grid import) data before August 1997

  • Last post 20 January 2021
  • Discussion is solved
morgan posted this 09 July 2020

Hi there,

Is it possible to get earlier data on generation (grid import) before August 1997? 

I've looked at the following.



These data sets are great, thank you for all the work you do.




Order by: Standard | Newest | Votes
EilyaTorshizian posted this 20 January 2021

Hi Phil and Morgan,

Thanks both. We need this for a Time Series econometrics analysis. If we get some reliable estimates, we will try to use the available metered data. But it's good to know that there are some possibilities to get more data from BigBag.Gnash, if more observations needed.



morgan posted this 20 January 2021

The import data at the node I was after started 1/04/1984



Phil Bishop posted this 20 January 2021

Hmmm, there really is a file called Gnash.exe in the folder at https://www.emi.ea.govt.nz/Wholesale/Datasets/_AdditionalInformation/ArchivedDatasets/201310_CentralisedDatasetFinal/CentralisedDataset/HalfHourly but you can't see it using a browser because .exe files are not one of the file types we allow to be surfaced in a browser. FWIW, I can see the .exe file using Azure Storage Explorer to view the EMI datasets storage container.

But all is not lost - see item 2 at https://forum.emi.ea.govt.nz/thread/new-access-arrangements-to-emi-datasets-retirement-of-anonymous-ftp/ for an alternative way to browse and download files from EMI datasets, using Azure Storage Explorer as it happens.

As noted in an earlier post above, the CDS was discontinued in October 2013 and it pre-dates the work of the Market Analytics team. Morgan notes above that BigBag.Gnash contained exactly what he/she was after. Whether or not that included metered grid data prior to Aug 1997, I couldn't say. TBH, I would doubt that it did - if the Authority held such data and we felt we could vouch for its reliability, we would've included it in https://www.emi.ea.govt.nz/Wholesale/Datasets/Metered_data.

I hope this helps.




Eilya Torshizian posted this 20 January 2021

Hi Phil,

We are looking for monthly/annual generation by fuel type and by node (or regional council). I will be able to extract this data from generation output by plant. That goes back to 1997m08.  This post is relevant because we were hoping to have some more historic data.

I couldn't find the Gnash.exe file in the folders - as suggested in the Gnash tutorial. Is it possible for you to either provide the link for Gnash.exe or let me know the timeframe covered by BigBag.Gnash?



morgan posted this 13 July 2020

Thanks Phil,

BigBag.Gnash had exactly what I needed.

Phil Bishop posted this 09 July 2020

The short answer is no. We hold no metered grid data in our data warehouse pertaining to the period prior to what is published on EMI at the links your have provided.

That said, if you rummage around in the old CDS folder, you might find something that meets your needs. The CDS pre-dates the efforts of the Market Analytics team; hence, we are unable to give you precise details about what you'll find therein.