Lag in real-time API data

1.8K Views
Last post 23 January 2019

icepicknz posted this 01 February 2018 - Last edited 12 February 2018

Hi,

When I compare the data on ElectricityInfo and that of the EMI API, it would appear the API is 25minutes delated.

For instance if I look at the following URL for my pnode KMO0331 right now at 6:45pm, I get the following data

{"interval":"01-FEB-2018 18:20","interval_datetime":"2018-02-01T18:20:00","five_min_period":5,"isDayLightSavingHR":0,"pnode":"KMO0331","load":11.442,"generation":0,"price":174.75}

However I get more realtime data if I look athttps://www1.electricityinfo.co.nz/prices#!#search_form%255Brun_types%255D%255B%255D=F&search_form%255Brun_types%255D%255B%255D=T&search_form%255Brun_types%255D%255B%255D=A&search_form%255Brun_types%255D%255B%255D=I&search_form%255Bmarket_types%255D%255B%255D=E&search_form%255Bmarket_types%255D%255B%255D=R&search_form%255Bnodes%255D%255B%255D=KMO0331&search_form%255Bdate_from%255D=2018-01-11&search_form%255Btp_from%255D=1&search_form%255Bdate_to%255D=2018-01-12&search_form%255Btp_to%255D=48&search_form%255Btoday_and_tomorrow%255D=true

When I noticed the problem, the API was still showing a spot price of $121 where as electricity info website was showing it at $174, the data was a good 15-20 minutes delayed which is no good for automation of generation and turning off power hungry devices etc.

Is it the intention for the API to be so lagged behind; is there a more accurate way of getting more up to date data?

I used to be able to use https://www.electricityinfo.co.nz/comitFta/five_min_prices.download?INchoice=SEL&INdate=31/12/2016&INgip=KMO0331&INperiodfrom=1&INperiodto=50&INtype=Average however that link no longer works.

#Permalink
0
0
0

API

Real-time prices

6 Comments

Order by: Standard | Newest | Votes

Admin

Phil Bishop posted this 01 February 2018 - Last edited 13 February 2020

I agree that it shouldn't be so difficult to get easy access to real-time prices, or any electricity prices for that matter. In fact, we've had this conversation before - see https://www.emi.ea.govt.nz/Forum/thread/timeliness-of-real-time-pricing-data.

Our EMI API is always going to lag behind real time but the lag should only be about 15 minutes. The system operator sends us all SPD case files in batches at 10 minute intervals. By the time we extract the real-time prices and get them to the API, we're generally about 15 minutes behind real time. I will investigate to see why that has slipped to the 25 minutes you're observing.

We also obtain all wholesale prices directly from the WITS SFTP server in CSV format and load them into our data warehouse. We plan to redirect the EMI API to that database but haven't got to it yet. That should speed things up a little but will probabaly still be about 10 minutes behind real time. Contact Dan at wits@nzx.com, the service provider responsible for publishing prices, if you want access to the WITS SFTP server.

That said, not everybody wants to maintain databases and mess around loading CSV files. Prices ought to be available via an API in a timely fashion. It is my understanding that the WITS redevelopment plan envisions prices being published via an API, but enabling that functionality is some years away.

You should also be aware that as part of the WITS redevelopment, the old Free-to-Air website has now been replaced with the WITS Data Hub at www.electricityinfo.co.nz. I'm not sure if that explains why the link you posted above no longer works. In any event, you should not hesitate to contact wits@nzx.com if you find something is not as it should be on the WITS site.

I hope this helps.

Phil.

#Permalink
0
0
0

icepicknz posted this 01 February 2018 - Last edited 23 January 2019

Thanks Phil,

I have reached out to wits and will see what they say.

A bit of background. I've got a fully automated house, power and appliances can be controlled from my smart hub. Yesterday I received a call from Flick saying they are running a beta for home generators (solar) so I signed up. When dealing with spot prices, once needs to be weary of the possible fluctuations in cost, so I wrote a quick condition on my house to turn off all lights and appliances if spot prices were over $200/MW, but to also warn me at intervals before hand by changing the color of lights in the house.

In addition to the above, I have a Tesla model x arriving in March; being able to see spot prices and control the charge rate of my car from 32A when spot prices are low, to 4A when high and off when real high is quite useful when you consider i'll be adding 100kw of power to my car for long trips.

Lastly I have 5kw of solar generation and am looking to add another 5kw, plus a powerwall 2 in April (been on the waiting list over a year now), being able to direct energy to the grid when spot prices are high rather than when they are low works out more economical. I'd prefer to be charging my car at <5c a kw and then turning charging capabilities off and sending that power to the grid when its 40c a kw and earning money from my investment.

Heres hoping I can find a solution that works for my needs, however the API and information provided thus far is certainly a plus and I'm very grateful.

Many thanks

Barry Murphy

#Permalink
0
0
0

icepicknz posted this 08 February 2018 - Last edited 08 February 2018

Hey Phil,

I’m unsure what the different between PRS and RTP are and haven’t been able to find anything online, can you explain?

The reason I ask, my flick App says my current power estimate is 29.3c per kw, so about 18.5c per kw of spot price

The EMI API says its 12.195c per kw

The WITS website shows the PRS as 18.7c per kw and the RTP as 12c per kw.

Just not sure which one to trust; i wonder if i build my smart app to look at and trust flicks data, or if i look direct on the EMI (Which i know is 25mins delayed) or build into WITS SFTP :)

Many thanks

#Permalink
0
0
0

dewitpj posted this 22 January 2019 - Last edited 23 January 2019

Hey Barry,

Sorry to dig up an old thread, but did you ever resolve this ?

P.S. How is the X going ?

Thanks,

Pieter

#Permalink
0
0
0

icepicknz posted this 22 January 2019 - Last edited 23 January 2019

Unfortunately not; i just went with its not possible to trust the real time data as it was me notifying EMI it was broken and sometimes a few days later :/

X is going great, loving it

#Permalink
0
0
0

dewitpj posted this 23 January 2019 - Last edited 23 January 2019

B?gg?r - thanks anyways

#Permalink
0
0
0

Find discussions by tag

See all

Lag in real-time API data

Categories

Find discussions by tag

This week's high earners

Popular discussions