EMI features and tips Discussions

EMI retirement and Electricity Authority data and insights hub

2025-09-29T03:05:50.5470000

The Authority will soon begin to decommission the EMI platform. A new ‘Data & insights’ hub on the Authority website will replace EMI and become the single place to go for electricity market data, tools, and insights on New Zealand’s electricity markets. See also a recent news item about the new Data & insights hub.

The Data & insights hub can be accessed directly from the menu items across the top of the Authority website.

Some EMI content is already available on the new Data & insights hub; we will migrate the rest across in the coming weeks:

APIs
Datasets
Instructions on how to programmatically obtain datasets

Other content from the website has been relocated to the new Data & insights hub:

Trading conduct reports – keep informed about how the markets are performing in our weekly and quarterly reports
Market reviews – explore in-depth reviews into unusual events or outcomes
Eye on electricity explainers – learn how electricity and the sector works to keep the lights on
Security of supply snapshots – understand how secure our electricity supply is every week.

Today we have disabled the ability to create an EMI login and we've begun to decommission the EMI forum. It will no longer be possible to create a discussion on the forum. We will also soon decommission the EMI mailbox.

If you have any feedback or suggestions on the ‘Data & insights’ hub, we would love to hear from you! Please email info@ea.govt.nz.

We will continue to update this post with further information and the timing of upcoming changes.

Accessing EMI datasets with Azure Storage Explorer

2020-04-27T22:24:48.0370000

1) First Download and Install 'Microsoft Azure Storage Explorer' by following the instructions on the page https://azure.microsoft.com/en-us/features/storage-explorer/.

2) Open the 'Microsoft Azure Storage Explorer' application, and select the Open Connect Dialog button (Plug Icon):

3) The Connect to Azure Storage interface will show options for connecting to the storage account. Choose the Use Shared access signature (SAS) URI option then select Next:

4) Copy this string: https://emidatasets.blob.core.windows.net/publicdata?sv=2019-12-12&si=exp2022-03-31&sr=c&sig=m2kSNI61k%2BmyHl1O84EdFlaXbgRMve%2BNByuDIzoMMXg%3D

Paste it into the URI:

Please not that the SAS token will expire, so this value may change over time.

Select Next

5) You will be shown a Connection Summary, check the values and select Connect:

6) You should now be presented with access to the Blob Container and the folders:

Demand override when working in vSPD 3.10 and GAMS

2024-06-25T01:54:38.5500000

Kia ora koutou, I'm new to vSPD and haven't used GAMS since the 90s but have been impressed with what I have seen so far workign with vSPD and GAMS in the last couple of months. I'm wondering if there is an example of the format for demand override GDX file? We exported i_tradePeriodNodeDemand into Excel, removed the columns for the nodes we weren't changing, changed the values for the nodes we were changing and then imported this data as ovrd_tradePeriodNodeDemand into a new GDX file. We pointed to this using vSPDsettings.inc but are getting errors:

Symbol not in GDX

I think I'm misunderstanding how to specify overrides so any help is very welcome. Nga mihi nui in advance for any help! Noho ora mai, Mike

ICP Connection API V2 is unavailable

2024-07-15T04:40:59.0470000

As at 16:37 15 July 2024 the developer portal at

https://emi.developer.azure-api.net/api-details#api=ICP-connection-data-v2

is returning a 500 Internal Server Error with

{
    "Code": 500,
    "Message": "API is currently unavailable to process this request. (R)"
}

Is there a status page we can consult to determine if the API is up?

Regards

Henry Maddocks

ICP connection data v2 - Meter Type 'Undetermined'

2024-03-27T03:25:12.3600000

When attempting to access ICP connection data via the EMA API, some results return with Meter Type 'Undetermined'. I know that several of these are smart meters. What is the reason for the 'Undetermined' category?

Thanks for your help

Pricing_20231025.gdx not available

2023-11-12T19:57:40.9700000

Hi,

I wonder when the Pricing_20231025.gdx file will be available on the daily GDX files webpage.

Regards,

Ziming Guan

Issue applying override

2023-08-17T22:52:58.1130000

Hi there,

I'm a newbie vSPD user trying to create an override for a hypothetical solar injeciton at HLY220 node. Attached is the CSV file I used to create the override.

The most recent job I tried to use it on is called "try try again". When doing the run it appears to completely ignore the new offers, so clearly I'm missing something obvious. Any help is appreciated.

EMI datasets - Access via a storage client (Azure Storage Explorer)

2023-04-03T21:56:33.6230000

Re: the below, i may have missed it elsewhere, but is there an updated SAS as the below one has expired ? (Thanks in advance)

2. Access via a storage client (Azure Storage Explorer)

If you’d like to browse through the datasets as you’ve previously done with an FTP client, you can now use Azure Storage Explorer and connect to EMI datasets using this Shared Access Signature URI:

https://emidatasets.blob.core.windows.net/publicdata?sv=2018-03-28&si=exp2019-12-31&sr=c&sig=RgEr3fnUCRgCg%2FGc%2BYus0OJHXpWQBZvUPpIDxsOtJQE%3D

https://emidatasets.blob.core.windows.net/publicdata?sv=2018-03-28&si=exp2020-12-31&sr=c&sig=7AB6rWqiWNl87RgAck6U9oOUMhmp6EOZXm1mfAbq%2Bvc%3D

https://emidatasets.blob.core.windows.net/publicdata?sv=2019-12-12&si=exp2022-03-31&sr=c&sig=m2kSNI61k%2BmyHl1O84EdFlaXbgRMve%2BNByuDIzoMMXg%3D

Edit: Updated SAS URI
https://emidatasets.blob.core.windows.net/publicdata?sv=2020-08-04&si=exp2023-03-31&sr=c&sig=TmXW68yI9Z2PUnheGD6PAy3c8cTdoug1tY7UrDMWuVE%3D

Date format change in metered data files (from march 2022)

2023-01-29T21:54:57.8670000

Hi there,

Interested to understand the rationale for change of date format for the metered data from March 2022 onwards.

Prior to then it was DD/MM/YYYY, and after it was YYYY-MM-DD.

Duplicates in SPD case files

2022-12-14T05:26:19.6330000

I'm trying to collate the 30-minute pricing data at each node, and I've noticed that on occasion the same INTERVAL is present in more than 1 SPD case file - as an example, between cases 61012022121905054 and 61012022121905055. I'm just hoping to clarify how to correctly calculate the 30-minute settlement price from individual cases, particularly where there are two cases for the same INTERVAL, or, conversely, where there are no cases for a 5-minute INTERVAL.

Cheers,

Jordan.

Several EMI reports going offline for a few weeks

2022-04-12T04:45:25.3130000

On 20 April 2022 the EMI reports listed below will go offline. Some will be offline for a matter of days while others will potentially be off for some weeks.

We have been transitioning our data platform from SQL Server databases to a Databricks-based Delta Lakehouse over the past several months. Consequently, all of the procedures used to drive the EMI reports have had to be rebuilt and pointed at the Delta Lakehouse. After Easter we are finally going to pull the plug on the SQL Server machine. This means that the last few EMI reports that have yet to be rebuilt will go offline.

In the coming weeks, we will be working to get them back up and operating. Some may even be switched over before the SQL Server machine is decommissioned. First to be restored from the list below are the two win-back reports in the Retail section.

We can prioritise the order of the rest based on demand so if there are reports in the list below that you want to see sooner rather than later, please let us know by posting below.

Wholesale

Wholesale price map
Business demand trends (also listed in Retail)
Vertical integration trends
Binding constraints - binding constraints are available now as a dataset

Forward markets

Bid-ask spread trends
Market depth (this report hasn't been updating for some time but will restored in the future)
Implied spot price (this report hasn't been updating for some time but will restored in the future)

Dual listed in both Forward markets and Wholesale

Location factor map
Location factor calculator
Physical wholesale position

Retail

Consumption data request trends
Residential consumption (map and chart)
Residential consumption league table
Consumer choice
Market concentration league table
Residential savings league table
Business demand trends
Consumer choice league table
Entry and exit
Residential consumption trends
Win-back indicator summary
Win-back indicator trends

New access arrangements to EMI datasets (retirement of anonymous FTP)

2018-10-16T04:31:38.8900000

The old anonymous FTP access to the EMI datasets has been replaced by REST-based access to Microsoft Azure storage. The existing FTP access and testing sites will be disestablished on 8 November 2018. Please ensure you have adjusted any scripts you rely on before then.

These options are being provided following the testing and feedback received here and via email. Thank you to those users who participated in testing and provided feedback.

Four methods to access EMI datasets are supported

1. Web-based access via www.emi.ea.govt.nz

You can still use the EMI website to browse content as you always have. The main menu categories have a 'datasets' option in the drop-down that lets users browse folder content and download files one at a time. Each folder will usually be supported by a short paragraph describing its content.

This method is best for ad-hoc downloads of a few files every now and again and provides the most descriptive information to support the data.

2. Access via a storage client (Azure Storage Explorer)

https://emidatasets.blob.core.windows.net/publicdata?sv=2020-08-04&si=exp2023-03-31&sr=c&sig=TmXW68yI9Z2PUnheGD6PAy3c8cTdoug1tY7UrDMWuVE%3D
https://emidatasets.blob.core.windows.net/publicdata?sv=2021-10-04&si=exp2024-03-31&sr=c&sig=fH6rIJPLMPtmt37cC6CZk44UDLs0E0C9Sy695KQCxlo%3D

Edit April 2024: Updated SAS URI for 2024 onwards
https://emidatasets.blob.core.windows.net/publicdata?sv=2021-10-04&si=publicdata&sr=c&sig=f034UWz1xmMbk89jd76zY0M%2BwycFDhhumejUrjqlfIw%3D

This method is best if you are used to using a client or want to manually download whole folders or a large set of files.

Figure 1: Add the URI above in the connect dialogue box

The access token included in the URI above does expire and will be updated in the future. The expiry date is denoted as part of the URI “si=exp2019-12-31” we will update users by posting on the forum prior to this expiry date.

3. Shell/Command line access (AzCopy)

Ideal for both one offs and scripting. Instructions and download links here. Use the second half of the URI above to connect ie:

Example 1: Download a file

azcopy.exe copy "https://emidatasets.blob.core.windows.net/publicdata/Datasets/Wholesale/MappingsAndGeospatial/NetworkSupplyPointsTable/20230404_NetworkSupplyPointsTable.csv??sv=2021-10-04&si=publicdata&sr=c&sig=f034UWz1xmMbk89jd76zY0M%2BwycFDhhumejUrjqlfIw%3D" "C:\Temp"

Example 2: List files

azcopy.exe list "https://emidatasets.blob.core.windows.net/publicdata/Datasets/Wholesale/MappingsAndGeospatial/NetworkSupplyPointsTable/??sv=2021-10-04&si=publicdata&sr=c&sig=f034UWz1xmMbk89jd76zY0M%2BwycFDhhumejUrjqlfIw%3D"

4. Full programmatic access

If you want to automate access to the datasets, or you already have scripts that do this, they will need adjusting. The BLOB storage endpoint to use is the same as above or you can just directly access the storage container via https://emidatasets.blob.core.windows.net/publicdata?restype=container&comp=list.

This method is best if you want to set up scripts to automatically download files into your own system.

The date modified of files can be used to find the latest files. This process will normally be fine, although some intermediate files may get refreshed multiple times. In addition, we may infrequently regenerate entire sets of files as we improve the data quality and align formats. We’ll aim to notify you on this forum of any such changes ahead of time.

Example 1: Access more list information

Each list request will contain a maximum of 5000 blobs. If there are more than 5000 blobs a marker value is included in the ‘NextMarker’ element at the end of the XML response. To return the next set of results, pass the value returned in the NextMarker tag as the marker parameter in the request URI.

https://emidatasets.blob.core.windows.net/publicdata?restype=container&comp=list&marker=2!144!MDAwMDY0IURhdGFzZXRzL1dob2xlc2FsZS9CaWRzQW5kT2ZmZXJzL09mZmVycy8yMDE2LzIwMTYxMjEwX09mZmVycy5jc3YhMDAwMDI4ITk5OTktMTItMzFUMjM6NTk6NTkuOTk5OTk5OVoh

Example 2: Access to a folder

Alternatively, you can narrow your search to a specific folder by using the 'prefix' parameter in the request URI.

https://emidatasets.blob.core.windows.net/publicdata?restype=container&comp=list&prefix=Datasets/Wholesale/BidsAndOffers/Bids/2018

Example 3: Access to a file

https://emidatasets.blob.core.windows.net/publicdata/Datasets/Wholesale/BidsAndOffers/Bids/2018/20181015_Bids.csv

Further assistance is available via the links below:

REST reference to access AZURE BLOB storage: https://docs.microsoft.com/en-us/rest/api/storageservices/Blob-Service-REST-API

The following articles offer specific guidance in your language of choice:

API suddenly returning no data?

2021-10-27T05:43:51.0000000

Hi there,

All of a sudden I am getting errors (see below) and am wondering what has changed on server side?

Do I need to update my API key?

Thanks

Oct 27 18:37:18 farm0 Node-RED[25917]: 27 Oct 18:37:18 - [error] [function:Extract price for ISL0661] TypeError: Cannot read property 'DollarsPerMegawattHour' of undefined

Oct 27 18:37:48 farm0 Node-RED[25917]: {

Oct 27 18:37:48 farm0 Node-RED[25917]: payload: [],

Oct 27 18:37:48 farm0 Node-RED[25917]: topic: '',

Oct 27 18:37:48 farm0 Node-RED[25917]: _msgid: '43e7a779.d760c8',

Oct 27 18:37:48 farm0 Node-RED[25917]: headers: {

Oct 27 18:37:48 farm0 Node-RED[25917]: 'cache-control': 'no-store',

Oct 27 18:37:48 farm0 Node-RED[25917]: pragma: 'no-cache',

Oct 27 18:37:48 farm0 Node-RED[25917]: 'content-length': '2',

Oct 27 18:37:48 farm0 Node-RED[25917]: 'content-type': 'application/json',

Oct 27 18:37:48 farm0 Node-RED[25917]: expires: '-1',

Oct 27 18:37:48 farm0 Node-RED[25917]: 'arr-disable-session-affinity': 'True',

Oct 27 18:37:48 farm0 Node-RED[25917]: 'x-content-type-options': 'nosniff',

Oct 27 18:37:48 farm0 Node-RED[25917]: 'x-frame-options': 'deny',

Oct 27 18:37:48 farm0 Node-RED[25917]: 'strict-transport-security': 'max-age=31536000; includeSubDomains; preload',

Oct 27 18:37:48 farm0 Node-RED[25917]: 'x-xss-protection': '1; mode=block',

Oct 27 18:37:48 farm0 Node-RED[25917]: 'content-security-policy': "script-src 'self'",

Oct 27 18:37:48 farm0 Node-RED[25917]: date: 'Wed, 27 Oct 2021 05:37:48 GMT',

Oct 27 18:37:48 farm0 Node-RED[25917]: connection: 'close',

Oct 27 18:37:48 farm0 Node-RED[25917]: 'x-node-red-request-node': '5922c7cc'

Oct 27 18:37:48 farm0 Node-RED[25917]: },

Oct 27 18:37:48 farm0 Node-RED[25917]: statusCode: 200,

Oct 27 18:37:48 farm0 Node-RED[25917]: responseUrl: 'https://emi.azure-api.net/real-time-prices/?$filter=PointOfConnectionCode+eq+%27ISL0661%27',

Oct 27 18:37:48 farm0 Node-RED[25917]: redirectList: []

Oct 27 18:37:48 farm0 Node-RED[25917]: }

API 101

2021-08-24T02:21:45.0370000

I now have access to the the ICP connection data API however I'm new to all this and have no idea how to submit a request I need API 101 help.
I have a few thousand addresses that I want to get the ICP codes for. Is there an example of a script that I can submit batches of say 25 addresses at a time to get back their ICP codes?

Any guidance much appreciated.

Warm regards

Lawrence

Potential for EMI connectivity issues on Tues evening, 15 Dec 2020

2020-12-11T03:18:17.3000000

On 15 Dec the Authority will be undertaking some infrastructure upgrades. While this work should not impact EMI, it is possible that something might go awry.

The Market Analytics team will take a look to see if this work has any impact on EMI processes on Tuesday evening. Worst case scenario is that if something were to fail, it might be Wednesday morning before we get it restored.

Thanks.

COVID-19 Authority and EMI update

2020-03-24T00:22:25.1030000

Hi all

Please see below for a copy of the COVID-19 Authority update that appeared in Market Brief this morning. As noted in the update, the Authority, including the EMI team, are like everybody else facing unprecedented challenges and pressures as we accommodate different working circumstances.

The EMI team will continue to keep the EMI reports and datasets up to date, respond to EMI forum posts, and answer emails arriving in the EMI inbox. However, as the team juggle work with family and personal needs, there may be times when we are unable to respond as promptly as we usually do.

We ask that you continue to send us your questions and comments, even if our response takes a little time. But most of all we ask that you do all you can to stay healthy and look after those around you.

Phil

COVID-19 Authority update

We are all facing new challenges as we move into an Alert 4 environment. We appreciate there will be pressures and we need to accommodate these different working circumstances.

The Authority is working remotely and we will continue to operate in our role as the regulator of the electricity industry. Our systems are set up and we have all the right technology to work from home and connect with each other and our stakeholders.

Our people are our priority and we need to make sure they are doing everything they need to do right now to protect themselves and their families, and to plan for living in an Alert 4 environment.

We will continue to work with our service providers to make sure they have everything in place to manage this extraordinary situation and to discuss alternative audit and reporting requirements.

We do not want to add to the stress on businesses or individuals right now. We will take a practical approach to support the system and give assurance to the industry and to all New Zealanders.

We will continue to carefully monitor the market and respond if need be. We have the right systems in place to manage any situation.

Staying connected

It’s important we stay connected and we continue to communicate key updates including work progress, consultations and other important information. The latest updates are listed below.

We will continue to use Market Brief, our website and social channels to stay connected and send updates.

In the meantime, please take care and email us if you have any questions.

E: info@ea.govt.nz

E: communications@ea.govt.nz

ICP ConnectionData - Too many requests

2019-10-03T04:39:37.2900000

Hi team,

I'm accessing the EMI ICP connection data interface. Given the caution around excessive use of the interface, I've structured my approach to do some JSON parsing in between calls and this slows things down to around 60 ICP's per minute, seemingly well below the 250 calls per minute threshold; yet I'm still getting the timeout message of 'Too many request'. Is there an overall volume limit that also applies that is kicking in here?

PS. I'm using the single ICP call.

Want to be able to filter by pnode when requesting RTP

2019-05-29T23:34:13.7270000

Getting realtime prices for the past 24 hours takes approximately one minute, and I wind up throwing away data for all but 1 of the 215 grid points. Is anybody else interested in the ability to query the API with a filter on pnode?

Column definitions and time zone information for real-time prices API

2019-05-25T03:43:00.4570000

Hi,

I am just starting out experimenting with the EMI Real Time Prices API and I have two questions:

1) Where can I find a definition of each of the columns returned by a call to the Real Time Prices API ? I get the following columns returned but it would be useful to know whether my assumptions about them are correct:

interval - What time zone is this in, is it UTC See 2) below ?
interval_datetime - What time zone is this in , is it UTC See 2) below ?
five_min_period - What does this mean, is it the 5-minute segment of the hour (ie there are values 1 to 12 ) ?
isDayLightSavingHR - What does the HR indicate ? As at 25May2019 (ie Daylight Savings is NOT in force) every row shows a value of <zero>. I would expect to see value of 1 when New Zealand is in Daylight Savings, but I notice that data returned for mid-Jan 2019 still shows a value of <zero>
pnode - This looks to be the power node. Is this correct ?
load - What scale is this / Megawatts, Gigawatts ?
generation - ??????
price - I assume that this is NZD, but again, $ per what ? (No pun intended)

2) What is the timezone being used for interval and interval_datetime?
When I ran the following query against the Real Time Prices API at 2019-05-25 15:30:
https://emi.azure-api.net/rtp/?$filter=interval_datetime gt datetime'2019-05-24T00:45' and interval_datetime lt datetime'2019-05-25T15:15'

the latest value I see returned for interval_datetime is '2019-05-25T01:25:00'. Is this simply a delay in making prices available or is the value offset by 14 hours for some reason ?

Any help or guideance would be appreciated.

Thanks.

Dave.

EMI ICP connection data API response delays – 10 Mar 2019

2019-03-10T21:28:06.3100000

Hi there,

ICP connection data API is apparently taking a signficant time to return for each polling attempt (anecdotally about a minute). Is there a currently known performance issue in place?

We have also taken the step of checking with Jade & there is not any performance issues from their side.

regards,

Report load timeouts occurring – 15 Jan 2019

2019-01-15T02:18:50.4770000

Anyone else struggling to even get to a dialogue page? I keep getting timed out before I have even requested any data!

Reminder: FTP access to EMI datasets will be removed on 8 November

2018-10-31T20:54:02.9670000

This is a reminder that FTP access will be removed in one week on 8 November. If you have automated scripts retrieving data via the FTP please modify those before this time. For information on the new access arrangements please see our earlier post.

https://www.emi.ea.govt.nz/Forum/thread/new-access-arrangements-to-emi-datasets-retirement-of-anonymous-ftp/

Important message for EMI FTP users

2018-08-17T04:27:09.4730000

Towards the end of September 2018, our EMI FTP server for direct access to EMI datasets will be decommissioned. Nearer the time, we will advertise on this forum the precise decommissioning date. We will also provide guidance on how to gain access to the Azure storage account that will replace the EMI FTP server and fileshare.

In the meantime, if you have any questions about the decommissioning of our EMI FTP server, please post to this discussion.

For quite some time now, we have been planning to migrate our data warehouse and EMI platform to Microsoft Azure, a public cloud facility. The challenges in doing so have been many and the hurdles have been high. But we're almost there and we can see the light at the end of the tunnel.

The following options provide high-level details on a number of ways to access the outbound storage account. A few random files have been placed in the account solely for testing purposes.

Option 1: Direct manual storage account access

The storage is quite easily accessed publicly if you're familiar with Azure Storage. Navigating to this address will provide an XML document detailing the contents of the storage account:

http://datasets.emi.ea.govt.nz/publicdata?restype=container&comp=list

Any of the URL elements can be located and downloaded directly. For example:

http://datasets.emi.ea.govt.nz/publicdata/Market structure/20180630_MarketShareTrendsByNetworkReportingRegion.csv

Option 2: Scripted access

A small Powershell script (or Python or any other scripting language) can be used to download files from storage… essentially automating the Option 1 process above.

Option 3: Web-based access

A single-page website displays the contents of the storage account and provides links to each file.

https://maoutstoreprd.z8.web.core.windows.net/

Option 4: SAS token access

Use a tool like Azure Storage Explorer and connect using a Shared Access connection URL. For example:

https://maoutstoreprd.blob.core.windows.net/publicdata?st=2018-08-09T03%3A05%3A42Z&se=2020-08-10T03%3A05%3A00Z&sp=rl&sv=2018-03-28&sr=c&sig=oYh9vJ%2FJlaFO8YVu7dHqpKHGSLOPk2yL7WQVqlOT7II%3D

Option 4 will return an error at this stage as we have not generated the necessary tokens. In fact we've yet to decide whether we will make this option available at all. It's in the mix as an option to connect to the storage account using an Azure Storage explorer tool. The SAS token is a Shared Access Signature which grants read-only rights into the storage for a fixed period of time to anyone with the URL.

Planned EMI outage, 4:30 pm Wednesday, 17 October

2018-10-16T04:07:37.3770000

We'll be taking EMI offline briefly at 4:30 pm on Wednesday 17 October. This outage is needed to make some database changes in our new environment. We should be all done and back up and running within an hour or two.

APIs and direct access to datasets will not be impacted. We apologise for the short notice on this occasion and any inconvenience this outage causes.

Planned outage of EMI and real-time APIs on Tuesday, 2 October

2018-09-24T23:08:50.9200000

On Tuesday, 2 October 2018 we’re making some changes to the infrastructure and processes behind the EMI website and the real-time pricing APIs.

EMI will be unavailable from around 12 noon on Tuesday, although full service should progressively return throughout the afternoon, with everything back up and running by Wednesday morning.

In addition to improving robustness and performance, we’ll be deploying a new release of the EMI website with enhanced features, making it easier for users to explore the depth of information behind reports.

The ICP connection data API and FTP site will remain unaffected. The real-time pricing API should not be impacted for long.

We apologise for any inconvenience this outage causes. Please email emi@ea.govt.nz if you notice any issues that remain unresolved on Wednesday.

Users who access the FTP server directly may also wish to follow the discussion over here. The current FTP server will continue to run in parallel with the new direct access arrangements for 30 days.

Adding a report to ‘my dashboard’

2017-11-29T04:55:30.7970000

‘My dashboards’ is a powerful feature we’ve introduced to EMI for registered users (you can register by clicking 'register' at the top of the screen or here).

The ‘my dashboards’ feature allows you to construct your own dashboards that are compiled from report views that you save. Developing your own customised dashboards on EMI allows you to quickly gain updated insights from the reports you’re especially interested in. The feature can also be used as a favourites list for frequently visited reports. These options make EMI easier and more efficient for you to use.

To add a report to a dashboard, select the button found at the bottom of any report. This will bring up a dialog box of settings.

To get started quickly, you can simply accept the defaults, enter a dashboard name, and hit add. Many report views can be added to a single dashboard.

There are four key features to consider when adding a report:

How would you like to handle date parameters?
How do you want to handle other parameters?
Which dashboard do you want to add the report to?
What do you want to call each report view on your dashboard?

1. How would you like to handle date parameters?

Three alternative situations for handling date parameters are explained below. These depend on the report and report parameter settings you have selected.

A. For reports with a single date parameter there are two options:

Use the default report date. This ensures that the report view in your dashboard will update as the dates change on the underlying report. Reports generally default to the latest date.
Save the selected report date to permanently capture a view of a point of time.

Example: A report showing monthly snapshot data

Try it yourself here.

B. For reports with time range parameters there are three options:

Use the default report dates. This ensures that the report view in your dashboard will update as the date range changes on the underlying report.
Save the selected quick range to always show this range (e.g. latest seven days) whenever the dashboard is viewed. This option is only available if the report being added is currently set to a quick range through the date selection parameter. This option is great when you want a dashboard to automatically stay up-to-date and the default date range does not work for you.
Save the selected report date range to capture a view of a point in time. This option allows you to save a report illustrating a market event or to save a specific view in your dashboard.

Tip: using dashboards to capture a view of a point in time works great if you have a temporary dashboard set up while you are looking into a topic, exploring a market event or developing a story.

Example: A report showing the latest seven days, i.e. the ‘latest seven days’ was selected when viewing the report before adding the report to the dashboard.

Try it yourself here.

C. For time series report with a time comparison added there are two options:

The time comparisons feature is an advanced feature that enables you to make time comparisons of a single time series for set lengths of time, e.g. compare this week to last week. These options are only relevant if you have added a time comparison to a report.

Save the selected quick range and relative time comparisons. An example might be adding a report view that always shows the latest seven days, compared to the two preceding weeks.
Save the selected report date range and time comparisons to capture a view of a point in time including the comparisons.

Example: A report showing a time comparison of the last seven days with the preceding two weeks

Try it for yourself here.

2. How do you want to handle the other parameters?

Handling the other report parameters is simpler than the date parameters. You have three options:

Use default report parameters
Save the parameters excluding the series filter selection (if available). This option allows you to save the parameters you have changed while letting the data and default chart settings drive the series filters.

Tip: combining this selection with a quick range selection means that your dashboard report won’t filter out the entry of new participants (if participants are the series in the report).
Save the parameters including the series filter selection. This option will only be available when you have changed the series filter parameter.

Example: A report where the series filter has been changed

3. Which dashboard do you want to add the report to? (or create a new dashboard)

You need to select a dashboard to add the report view to or you can add a name for a new dashboard. You are able to create up to five dashboards. The number of reports (or report views) in a dashboard is not currently limited although only eight reports are shown on each page.

4. What do you want the report view to be called in your dashboard?

Report names on EMI are rather generic to accommodate the range of views and insights that can be seen by changing report parameters. For example, showing a trend line for all of New Zealand or just one specific region.

You can add a specific name for the report and parameter selections you've saved to make it more meaningful to you in your dashboard. This naming feature is especially useful when you have added multiple views from the same report to the same dashboard (for example, perhaps illustrating activity in different regions).

See viewing ‘my dashboards’ and managing ‘my dashboards’ for more assistance on using the ‘my dashboards’ feature.

Change made to EMI FTP IP address

2018-03-12T00:07:58.9570000

Following the website outages at the weekend and some changes to the hosting arrangments, the EMI FTP site has a new IP address (202.27.77.215).

Users may need to update their firewall rules to accommodate this change. The old IP address is now obsolete.

We apologise if this change has caused any inconvenience.

ftp://emiftp.ea.govt.nz/

Planned EMI outage on 9 March 2018

2018-03-05T21:53:13.0170000

We’re planning to undertake maintenance on our websites on Friday. This means we expect our websites to be unavailable overnight from about 5 pm on Friday, 9 March, and potentially during Saturday, 10 March.

We aim to have the sites restored to normal operation as quickly as possible.

The affected websites are:

Electricity Market Information (EMI) website - this outage will also affect our real-time API products, but the ICP connection data API will not be affected
Electricity Authority website
What’s My Number website

Users can follow our Twitter account @electricityauth to be notified when our websites are back up and running.

Managing ‘my dashboards’

2017-11-29T04:55:20.2630000

Your dashboards are managed through the ‘list view and filters’ screen.

1. Rename or remove the entire dashboard

Buttons are available at the top of the page. A dialog box will appear where you can enter a new name or confirm that you want to delete the dashboard. There is no undo function so take care. You are currently able to construct and maintain up to five dashboards.

2. Rename or remove individual reports from a dashboard

To rename or remove an individual report from a dashboard you must be in the details view option rather than the summary view. Buttons exist to the right of each report name to rename or remove.

3. Adjusting the parameter settings for a report

There is no way for you to edit the parameters that were saved when the report was added to your dashboard. To achieve this outcome you can change the parameters on the report and add it again to your dashboard, and then delete the version of the report you no longer wanted.

4. Reordering reports in a dashboard

Reports are sorted in the order they are added by default – they can be filtered by tags and report types, or even sorted by the standard options available in list view. However, to change the custom order you must re-add the reports in the order you desire.

Tip: to reorder reports in a dashboard it's often easiest to add them to a second dashboard, in your new desired order, and then delete the first dashboard.

See adding a report to ‘my dashboard’ and viewing ‘my dashboards’ for more assistance on using the ‘my dashboards’ feature.

Viewing ‘my dashboards’

2017-11-29T04:55:13.2600000

There are three ways you can view items in your dashboard – a grid view, a list view, or by browsing individual reports in the set (with next and previous buttons). It’s up to you to choose which view works best for the content you’ve added, and the device they are on (desktop or mobile). The EMI website uses browser cookies to remember which view you last used for each dashboard and browser/device and will open in this view when you return.

1. Grid view

The grid view operates much as you expect a dashboard to – it displays a range of content visually on screen in panes of up to eight reports. Grid view allows you to quickly scan a range of content you have selected without having to set the desired parameters each time you view the report. The custom names you gave each report view when adding them to your dashboard will be displayed.

On narrow screen widths (i.e. a phone) grid view isn’t available as it doesn’t make much sense. In this situation, you’re better off drilling into a report and using the individual report view to browse a set of reports you have saved in a dashboard.

Reports in grid view will default to the order you added them to the dashboard.

2. List view and filters

The list view works great if you want a list of favourite reports that you use frequently with saved parameter selections. The custom names you gave each report view when adding them to your dashboard will be displayed.

Table reports won’t display in a grid view very usefully so are more likely to be added to a dashboard for use in a list view.

The list view can be set to display as a summary view or details view where each report’s description will also display. Reports will default to the order you added them to the dashboard.

This view is also used when managing my dashboards

3. Individual report view

When you drill into an individual report by clicking the heading you enter a view of a single report and have access to change parameters. In this view, you can browse using the next and previous buttons as with any set of reports defined by an EMI dashboard, a selected tag, filtered report types, and/or a selected report category.

The custom names you entered for a report when creating the dashboard will not show in this view as you also have the ability to change the parameters.

See adding a report to ‘my dashboard’ and managing ‘my dashboards’ for more assistance on using the ‘my dashboards’ feature.

EMI enhancements – Comparison feature for time series reports

2017-07-26T00:18:47.6170000

Context is an important part of interpreting and understanding data. A comparison of recent observations to another time period or similar event in history allows insights to be drawn from the event or time period and the circumstances that existed at the time.

We’ve added a feature that lets users compare one period of time against another for the same data series. In addition to providing historical context, it also provides a great way to explore the data. The comparison feature is only enabled on selected EMI reports and allows time comparisons of one, seven, thirty, or ninety days, or a calendar quarter or year.

More information on how to use this feature.

Example time comparisons

Increasing hydro risk curves (10% risk curve) 2015 - 2018
Final pricing demand* over the last 4 weeks (try changing the region type and then selections in the series filter)
Generation on Thursdays – 22 June to 20 July 2017 (try changing nodes in the series filter)

Example comparisons where the current measure will track against history, updating as new data becomes available

Wholesale prices 2017 compared to 2012 and 2016 (try adding additional years)
Total switches up in June on the last four years (try changing region or switch type)
May 2017 demand up in Central North Island (try Upper South Island and the impact on NZ demand - you may want to zoom in)
Changes in average PV capacity being installed by residential consumers
Final pricing demand this week, last two weeks, and same week last year

Figure 1: Final pricing demand this week, last week, and same week last year

* The SPD model dispatches offered generation to meet ‘final pricing demand’ and set the wholesale price. Wind generation is not dispatched in the SPD model but modelled as negative demand. This means that final pricing demand presented here is usually less than total demand. Reconciled demand which accounts for all offtake from the grid and networks becomes available around the middle of each month for the previous month.

How to use the EMI time comparison feature

2017-07-26T00:16:50.8770000

A time comparison feature for time series reports was introduced to EMI in July 2017. This post provides some additional information and tips on how to use this feature.

Adding a time comparison to a time series report

Time comparisons are only available time ranges of specific lengths. The date range drop-down on a report indicates what time comparisons are enabled. Not all comparisons shown in Figure 1 are available on all reports due to differences in the resolution of the data.

The comparison feature is enabled for any date range of the relevant length, not just the ‘latest’ available as a quick selection. To use a different base range rather than the latest for the report, users must use the custom range selection tool available at the bottom of the quick range options.

Figure 1: Date range selections with time comparisons

Step 1: Select a base date range

Select a base date range for a report that matches an enabled comparison setting (e.g. seven days). The ‘Add time comparison’ button will appear above the date range parameter.

Example: Wholesale prices at key nodes

Step 2: Add a time comparison

Clicking the ‘Add time comparison’ button will add a new parameter where users can select a start date for a day(s) comparison or year for a quarter or annual comparison.

This step can be repeated up to four times to add four comparison series to the base date range.

Day comparisons allow any single day to be compared to any other single day
Seven, thirty, and ninety-day comparisons are restricted to ensure that the same days of the week are compared across the series. (We may relax this restriction when we get some seven-day rolling averages included in the reports which will further enhance this feature)
Quarter comparisons are restricted to comparisons of the same quarter and years to any other year.

Figure 2: Date comparison parameters

Example: Benmore wholesale prices

Step 3: Change the series for the comparison

Only one series can be compared against history in a single report. If more than one series was selected when adding a time comparison, the comparison will be applied to the first series selected in the series filter. It is then a simple step to change the series filter which will have now become a single select parameter.

Example: Otahuhu wholesale prices

Other parameters in the report can be changed while retaining the time comparison settings. This feature is a powerful tool for exploring the data. However, clearly adjusting the base date range for the report will wipe out the comparisons.

Example: Change from generation in the South Island to nodes to see how generation patterns have changed over the last four weeks

Step 4: Share your insights

As with all EMI reports, the permalink captures the parameter settings enabling users to easily share the view they see with others. Permalinks are accessed via the link icon in the report settings next to data download.

When you drill through to the data report users will see that only have the base series selected. They will need to adjust the date range parameter to select a contiguous data series covering the time period they are interested in, or alternatively, download separate files for each time segment.

Setting a comparison report so that latest data tracks against history

A nice feature that time comparisons enable is a report where the latest data tracks progress against historical data as new data becomes available. This result is achieved by setting the base series so that the latest comparison series overlaps only part of the base series. When a report is set up this way, the most recent series (red) is highlighted and will appear slightly thicker than the historical series.

Figure 3: New Zealand wholesale prices, 2012, 2016, 2017

Example: New Zealand wholesale prices, 2012, 2016, 2017

EMI enhancements – improvements to series filter, report tags, and date range pickers

2017-04-27T23:10:58.7300000

We’ve just pushed out the latest enhancements to the EMI website. The most visible enhancements improve the usability of reports. However, we’ve also made some changes under the hood to improve performance and keep things better aligned and organised.

The usability enhancements include:

Series filters

On reports, your series filter selection now persists. This allows users to change other parameters while keeping the series items of interest selected. However, changing a parameter that results in a different set of series items will result in the series filter selection resetting. For example, if you change region type from zone to regional council where regions are the series, the list will clearly change.

Report tags

Report tags are now selectable. Clicking a subject tag below a report or dashboard will show a gallery view of all reports with that tag. This feature makes navigating or exploring a topic easier and helps to ensure users see the breadth of relevant information available. For example, the collection of switching reports.

Date range pickers

We’ve combined ‘from’ and ‘to’ fields into a date range picker to let users change both dates without having to wait for the report get the data and render after each date parameter was changed. The new date range pickers also includes a handy set of ‘quick range selections’ that are appropriate for each report to make it even faster to gain insights across time ranges.

We’ll continue to add content and improve the available features as time permits so please keep the feedback and ideas coming.

Planned EMI outage on 21 February

2017-02-17T03:52:44.3470000

The EMI website will be taken offline for a few hours on Tuesday 21 February 2017 at 6:00pm while we take care of some routine maintenance on the underlying SQL Server.

This also means that our real-time price API will be out of service during this planned outage.

We are hopeful that the maintenance work can be completed in a single session but if that is not the case, there will be a second outage on Thursday evening, 23 February 2017.

EMI website and real-time data API outage scheduled for 5 October 7pm

2016-09-29T02:25:48.3900000

We have some work scheduled on our data warehouse that will impact both the EMI website and the real-time data API product. We expect this outage to be between 1 and 2 hours starting at 7pm on Wednesday 5 October.

Access to the FTP server and the ICP connection data API product will not be affected.

We apologise in advance for any inconvenience this may cause.

EMI enhancements - New release of the EMI forum

2016-09-06T04:30:08.0070000

Today we’ve upgraded the EMI forum providing enhanced features to users

You can now:

subscribe to follow tags as well as categories and discussions ensuring you never miss out on a notification,
easily insert images (jpegs) into discussions and posts providing more flexibility and scope for meaningful discussion,
send private messages to other forum members (although we’d prefer you keep most content on the forum so others can benefit),
get suggested tags as you add them to a discussion reducing the opportunity for duplicate tags.

The 'EMI news' category is now a tag

The 'EMI news' category has been changed into a tag. This change means that we can keep each discussion in the relevant category aligned with the context of the discussion and separately tag it as newsworthy. If you were subscribed to follow the 'EMI news' category you have been migrated to follow the 'EMI news' tag. To find out what you follow click the 'following' heading in the forum menu.

Please note – if you follow a category or tag you will be notified of new discussions in that category or assigned that tag. This does not mean you will be notified of new posts in a discussion. To be notified of these posts, you will need to also follow the discussion itself if you are interested.

FTP server - users will need to migrate to a new URL

2016-08-23T22:55:37.9800000

We have installed a new FTP server to support the provision of data. Our old FTP server at ftp://ftp.emi.ea.govt.nz will be turned off on 23 September 2016.

The URL for the new FTP server is ftp://emiftp.ea.govt.nz.

If you use anonymous FTP to obtain data files directly from our FTP server, you will need to redirect your FTP client to the new URL before 23 September 2016. All other details associated with the use of anonymous FTP will stay the same.

If you download data directly from EMI Datasets using a web browser then you don't need to do anything - EMI Datasets is already pointing to the new FTP server.

Attaching files to forum discussions and posts

2016-06-29T02:40:10.8870000

You can attach files to discussions and posts on this forum.

Discussions:

To add a new discussion you must enter a title and some body text. You have an option of adding a tag to the discussion and can use an existing tag or add a new one. However, you must select a category for your discussion (subscribers to the category will receive a notification). A box will appear below the tags were you can browse to a file that will get uploaded. At this point, you can click the ‘Create discussion’ button.

To attach a second file, you must have first created the discussion and can then select the ‘Attach’ button from the right-hand side when viewing the discussion.

Posts:

Much like attaching a second file to a discussion, you can attach a file to a post after you’ve first added the post. Use the ‘Attach’ button from the right-hand side when viewing the discussion and associated posts.

Files types:

File types that can be attached are jpg, jpeg, png, gif, pdf, zip, txt, and csv.

The Authority assumes no responsibility for files posted by others on the forum. We would encourage you to exercise care when downloading files, and especially when opening zip files.

Images can be embedded in discussions and posts if the image is hosted online somewhere and has a URL. We find jpeg images constrained to a width of 700 pixels work great. We use style 'heading 4' for the figure titles.

Downloading data files with FTP

2016-05-30T04:44:07.0700000

It is a straightforward task to download the data behind an EMI Report. It is also easy enough to jump over to Datasets and find a file or two to download. We are often asked if it is possible to obtain data using FTP. The answer is yes.

Our EMI FTP server supports anonymous FTP. In fact, Datasets is nothing more than a browser-based view of the file structure on our FTP server.

At the command line, just type:

ftp ftp//ftp.emi.ea.govt.nz

You should not need to enter a username or a password. If you do get prompted for such details, use anonymous for the username and your email address for the password.

A list of commands to be used while in the FTP session can be found at http://www.computerhope.com/issues/ch001246.htm, https://technet.microsoft.com/en-us/library/bb490910.aspx, or http://www.cyberciti.biz/faq/linux-unix-ftp-commands. There are dozens of websites available with help on using FTP. Alternatively, type 'help' on the command line to see all available commands and type 'help command_name' for the description of a particular command.

Two commands you'll want to use are 'cd' to change directory and 'mget' to get multiple files. Also 'ls' or 'dir' to see what's in the directory you're currently in.

Finally, you can use Google to find numerous FTP applications to download and install on your computer if you don't like using command line applications.

EMI enhancements – New features on the EMI website

2016-05-27T04:30:33.8030000

Today we launched the refreshed look and feel of the EMI website and a suite of four new features, as outlined below.

Discussion and collaboration

The EMI forum is an online platform where a community of active contributors can ask questions about the data, materials and tools available on the EMI website. Users are encouraged to engage in discussions of a technical nature and will be supported by Authority staff members, who will play an active role in the forum and offer educational assistance.

EMI forum

Experimentation and analysis

vSPD-online is a web-based implementation of the Authority’s vectorised Scheduling, Pricing and Dispatch (vSPD) model. In vSPD-online, users can manipulate data and set their own parameters to examine hypothetical situations. For example, an industrial user could use vSPD-online to plan scheduled outages, because it would allow them to test their tolerance for changes in demand and wholesale prices.

vSPD-online

Streamlined information

In addition to receiving information through reports, datasets or FTP access, the new-look EMI website incorporates our application program interface (API) platform. The API platform includes the ICP connection data API (powering a form on the Authority’s corporate website) along with real-time pricing and real-time dispatch information. These APIs allow users to automate information flows by subscribing to feeds or requesting specific results. For example, an industrial user could use the information in our real-time pricing API to support the outage testing they undertook using vSPD-online.

API platform

Searchable information

We have also added a search facility to the EMI website to help users quickly and easily locate tools and data of interest.

The launch of these new features is a significant improvement in providing assistance to, and strengthening our relationship with, analysts and technicians in the electricity industry. We look forward to working with you and invite you to send an email to emi@ea.govt.nz if you’d like to provide feedback or ask questions.

Managing your EMI user account details

2016-05-23T04:07:59.5170000

You can manage your profile and user account details by selecting 'Edit' from the drop-down menu on the top right in the blue bar. Amongst other options you can update your username, email address, affiliation, and password here.