Timeliness of real-time pricing data

3K Views
Last post 10 July 2016

gleyland posted this 17 June 2016 - Last edited 18 September 2016

Hi,

I host http://www.nodewatch.co.nz/, a website for electricity users taking spot pricing, or small generators taking spot pricing that sends alerts to those users when prices pass through thresholds the users can set.

The signed-up users appear to be a mix of Flick customers, industrial users and small generators, all of whom wish to react rapidly when the price goes high, either by turning things off, or by turning generation up.

Recently, there have been a number of short-duration prices spikes, like the one on the second of June. This has caused two things: one, a flurry of sign-ups to nodewatch, and two, questions about the timeliness of the data. If a price spike lasts a single trading period, it's very important that the alerts get to the users rapidly.

(For example, see this discussion on Geekzone.)

Currently, I scrape five-minute prices from WITS free to air (and am *fully* aware of the difference between real-time prices and final prices, we don't need to resolve that here). There are a number of issues with this (scraping is awkward and slow), but the current issue is timeliness of the data: by the time I get the price from WITS about half a trading period has passed. I understand the EMI real-time data APIs are similarly delayed.

As I said, I understand real-time prices are not the same as final prices, but at this time, they are the best thing nodewatch users have to go on. Large generators in the industry have *plenty* of tools to help manage (and forecast) these prices. It would be really helpful to have timely data on real time prices available for the small players and customers to enable them to respond before it's too late.

Thanks!

Geoff

#Permalink
1
0
0

API

Real-time prices

Liked by

3 Comments

Order by: Standard | Newest | Votes

Admin

Phil Bishop posted this 22 June 2016 - Last edited 13 February 2020

It is useful to elaborate on several themes alluded to in Geoff’s discussion:

The short answer to Geoff's question is that there is no ideal solution at this time. But the good news is that we’re working on it.

The Authority is focused on the need to keep the Code and the various market operation services in a fit-for-purpose state, and contributing to our statutory objective. Several initiatives towards this end have already been incorporated into the Authority’s work programme. The Authority welcomes and encourages stakeholder engagement when it consults on its work programme so please do let us know if you believe our market service arrangements are not keeping pace with technological change or the evolution/maturing of the electricity market. Similarly, a process exists for anybody to propose an amendment to the Code.

The wholesale information and trading system (WITS) is just one of many service provider roles that the Authority contracts out. WITS is designed to support the flow of information between wholesale market participants and the system operator. It doesn’t cater well to the emerging ‘energy services’ community – these are firms looking to manage energy (including electricity) usage and procurement on behalf of consumers – or the small players and customers Geoff mentions. Not all consumers want to engage in the electricity market on a direct or frequent basis; some prefer to be more passive about it and outsource the direct engagement. As Geoff has noted, the WITS free-to-air website is clumsy to machine-read or scrape. If you’re a final consumer and all you want to do is look up a single real-time price every now and then, it will probably suffice. But it is an impractical platform from which to develop innovative new services. The Authority is keen to support initiatives that foster innovation, which in turn contribute to a more efficient and competitive market.

The Market Analytics team that maintains EMI sits within the Market Performance team at the Authority. As the team name suggests, our focus is on market performance and not in supporting the real-time operations – the Authority contracts with service providers to do that. Consequently, the infrastructure used to transfer real-time price and real-time dispatch case files to the Authority is outside of the market systems. This is why it takes about 10 minutes for the RTP and RTD case files to reach us. Our purpose in standing up these APIs was to assist the emerging energy services community in the absence of any other convenient method to obtain information, and to demonstrate the implementation and use of a low-cost, off-the-shelf technology to produce a world-class RESTful API.

As noted above, WITS is presently set up to cater to participants. However, anybody is able to request access to the WITS data portal; a faster and more automated solution than scraping the WITS free-to-air website. But there are a couple of hoops to jump through. Access to the WITS data portal requires prior approval of the Authority and agreement to the WITS use of system agreement, available on request from the WITS manager.

The story is not all bad, however. The Market Operations team at the Authority is working through a WITS upgrade project with the WITS manager. We have requested that simple, low cost, quick-to-deploy APIs such as the Authority’s RTP and RTD APIs be part of this upgrade so that the non-participant community is able to gain access to timely, relevant data on the same basis as participants, and thereby offer innovative services to end consumers. Unfortunately, a real-time price API is planned for the third phase of this project so the waiting will have to continue for some time yet.

Finally, it is useful to clarify a few points around the determination of final prices. The Geekzone discussion referenced above by Geoff had some commentators assuming final prices were an average, or a weighted average, of real-time prices. They are not. Final prices, the prices that wholesale participants ultimately pay or get paid, are determined ex post, usually within a day or two, and by a different run of the pricing model using a different set of inputs. See here for more information. The Authority has a programme of work that, among other things, is looking to introduce settlement based on real-time prices.

#Permalink
0
0
0

John.andrews.nzx posted this 08 July 2016

Hello Geoff,

The product owner for WITS is keen to hear from anyone with requests or comments on using WITS. The new data services, and the replacement for WITS FTA (to be named WITS Data Hub) and in development now.

Please feel free to contact the service providers at cm@nzx.com or wits@nzx.com.

John Andrews

Senior Energy Analyst

NZX