EMI datasets updated – NSP table, metering configurations, market structure and shape files

  • 1.1K Views
  • Last post 1 weeks ago
Matthew Keir posted this 20 December 2017

We've made some changes to existing datasets on EMI. These changes are outlined below. The existing files and file structures will be maintained until after the data for January 2018 is published in mid-February. Users will have to migrate any automated processes to the new file names and locations before the February data is published in March. 

1. Network supply points table

Updated filename and location and the addition of region ID (one column) for network reporting regions to assist users when combining NSPs with other datasets that we publish. 

Obsolete file and location (to be disestablished mid-February 2018):

New file and location:

2. Distribution price category code added to metering configurations file

Some participants asked us to add distribution price category code to the metering configurations file that we publish. We agreed that this would help reduce barriers to entry for participants and third parties by helping parties setup billing processes or match consumers to retail plans.

Obsolete file and location (to be disestablished mid-February):

New file and location:

3. Rename of market structure files

This is simply a tweak to file names and location.

Obsolete file and location (to be disestablished mid-February 2018):

New file and location:

4. Network region shape files

Relocating these files to make them easier to find.

Obsolete file and location (to be disestablished mid-February 2018):

New file and location:

Order by: Standard | Newest | Votes
Matthew Keir posted this 07 December 2021

Thanks for letting us know. This has now been resolved.

  • Liked by
  • msouness
msouness posted this 18 January 2018

Hi Matthew,

Do these locations reflect the FTP file locations?

Matthew Keir posted this 18 January 2018

Yes, the FTP and datasets on EMI are the same underlying files and structure. I have only provided the old and new dataset URLs above. However, if you browse the FTP you'll see the locations are already there.

Alternatively, if you change the front part of the URLs above to the FTP equivalent, then the URLs above will work on the FTP server. For example: substitute "www.emi.ea.govt.nz/Wholesale/Datasets" for "ftp://emiftp.ea.govt.nz/Datasets/Wholesale/".

msouness posted this 18 January 2018

Filenames appear to have "_" in one, and no spaces in the other?

Matthew Keir posted this 18 January 2018

Hi Malcolm,

Sorry for the confusion. We are migrating to a camel case for file naming convention across our data warehouse including all files exported to datasets. This change will take some time and usually occur when we are doing something else with that piece of code. We'll advise of these changes through the forum and give a period of time for users to adjust their process as above.

Might you be looking at the old file still?

For example:

Obsolete file and locations (to be disestablished mid-February):

New file and locations:

Does that make sense?

Matthew Keir posted this 22 February 2018

Update...

The old files and locations have now been removed. However, some participants asked for more time to adjust processes around the NSP table. To accommodate these users we will keep the old location for the NSP table until mid-March.

The new NSP table location is:

Matthew Keir posted this 26 March 2018

The old NSP table files and location have now been removed.

tship posted this 06 December 2021

Morning Matthew Keir re: Wholesale datasets: Network supply points (NSP) table, looks like there haven't been any new files since 21st November ... just checking this is as expected.

Thanks in advance ...

                 

mmoore posted this 01 May 2022

@https://forum.emi.ea.govt.nz/profile/matthew-keir/  

re: Wholesale datasets: Network supply points (NSP) table, looks like there haven't been any new files since 14th April ... just checking this is as expected.

 

Thanks in advance ...

 

 

 

tship posted this 02 May 2022

@Phil Bishop could you advise as see that Mathew has left ... thanks in advance

Guy Ross posted this 03 May 2022

Thanks everybody, the network supply points table is now being exported again.

msouness posted this 1 weeks ago

Right, what's the story with the Meter reports being limited only to ANZSIC codes and metering categories?

We need to get the meter register per content code and period of availability data back online in order to keep check on the MEP Data in the Registry, as errors in that space greatly impact retailer's abiltiy to bill and report energy to EDB's accurately.