New EMI API release – 1 March 2020, Version 2 of the ICP connection data API (ACCES quick wins)

  • This discussion is locked, please start a new discussion.
  • 2.1K Views
  • Last post 25 May 2020
Matthew Keir posted this 20 February 2020

Version 2 of the ICP connection data API is scheduled to go live on Sunday 1 March 2020.  The original ICP connection API will be deprecated but will continue to operate as it currently does so no systems relying on it will be interrupted. This original version of the API will be turned off in six months’ time on 1 September 2020. 

Subscribers to the API will be able to migrate to version 2 and take advantage of the additional information at their convenience. Migrating to the new version should be a simple change to the URL call to include (v2) and handling on the additional fields and improved error responses. Users can test the API on the API platform when v2 becomes available. 

These changes are being made as part of the Additional Consumer Choice of Electricity Services (ACCES) project and will be accompanied by changes to the process to request consumption data from retailers. Agents are reminded that they will need to sign up to the new terms of the registry transfer hub to use EIEP13s. The new terms and sign up process will be made available next week on the provide a service with electricity page.


Version 2 ICP connection data API release notes

The original ICP connection data API that was released in 2016 as part of the Authority's 'Retail Data Project (RDP)'.

This version (V2) was released on 1 March 2020 following further consultation in 2019 as part of the 'Additional Consumer Choice of Electricity Services (ACCES)' project. This version of the API includes additional fields along with some other enhancements.

'Get by ID' and 'Get by ID list' call changes

New fields released

  • "PropertyNameOrDescription" (string) in the Address object (called "Property Name" in the registry functional spec)
  • "InitialElectricallyConnectedDate" (string formatted as YYYY-MM-DD) in the Network object
  • "TraderSwitchInProgress" (boolean) in the Trader object
  • "ANZSICcode" (string) in the Trader object
  • "ProfileCode" (string) in the Trader object
  • "CertificationExpiryDate" (string formatted as YYYY-MM-DD) in the Meter installation object
  • "ComponentType" (string) in the Meter component object
  • "SwitchReadIndicator" (boolean) in the Meter channel object (called "Settlement Indicator" in the registry functional spec, but only indicates if the channel read may be needed in the switch process)
  • "NetworkParticipantName" (string) in Network object (included via lookup of "NetworkParticipantID" field, will repeat the ID if not found)
  • "TraderParticipantName" (string) in Trader object (included via lookup of "TraderParticipantID" field, will repeat the ID if not found)
  • "MeteringEquipmentProviderParticipantName" (string) in Metering object (included via lookup of "MeteringEquipmentProviderParticipantID" field, will repeat the ID if not found)

Moved fields

  • "ICPIdentifier" moved to first in the return order
  • "ICPStatus" moved to second in the return order

Renamed fields

  • "NetworkParticipantID" in the Network object renamed from "Network"
  • "TraderParticipantID" in the Trader object renamed from "Trader"
  • "MeteringEquipmentProviderParticipantID" in the Metering object renamed from "MeteringEquipmentProvider"

'Search' call changes

New fields released

  • "PropertyNameOrDescription" (string) in Address object (called "Property Name" in the registry functional spec)

Gating and response code changes

  • 400 'bad request' code included filtering ICPs not meeting the required format
  • Messages will be returned in the messages object with the appropriate code where available 

 'Get by ID' call - successful response sample (The 'Get by ID list' call follows the same format)

{

  "ICPIdentifier": "string",

  "ICPStatus": 0,

  "Address": {

    "PropertyNameOrDescription": "string",

    "PhysicalAddressUnit": "string",

    "PhysicalAddressNumber": "string",

    "PhysicalAddressStreet": "string",

    "PhysicalAddressSuburb": "string",

    "PhysicalAddressTown": "string",

    "PhysicalAddressRegion": "string",

    "PhysicalAddressPostCode": 0,

    "GPS_Easting": 0.086597829994870867,

    "GPS_Northing": 0.089592538769447838

  },

  "Network": {

    "NetworkParticipantID": "string",

    "NetworkParticipantName": "string",

    "POC": "string",

    "ReconciliationType": "string",

    "InitialElectricallyConnectedDate": "string",

    "GenerationCapacity": 0.59222133055867521,

    "FuelType": "string",

    "DirectBilledStatus": "string"

  },

  "Pricing": {

    "DistributorPriceCategoryCode": "string",

    "DistributorLossCategoryCode": "string",

    "ChargeableCapacity": 0.74166837179270262,

    "DistributorInstallationDetails": "string"

  },

  "Trader": {

    "TraderSwitchInProgress": false,

    "TraderParticipantID": "string",

    "TraderParticipantName": "string",

    "ProfileCode": "string",

    "ANZSICcode": "string",

    "DailyUnmeteredkWh": "string",

    "UnmeteredLoadDetails": "string"

  },

  "Metering": {

    "MeteringEquipmentProviderParticipantID": "string",

    "MeteringEquipmentProviderParticipantName": "string",

    "InstallationInformation": [

      {

        "MeteringInstallationCategory": 0,

        "MeteringInstallationType": "string",

        "CertificationExpiryDate": "string",

        "ComponentInformation": [

          {

            "ComponentType": "string",

            "MeteringComponentSerialNumber": "string",

            "MeterType": "string",

            "AMIFlag": false,

            "CompensationFactor": 0.68670529385750934,

            "ChannelInformation": [

              {

                "MeteringComponentSerialNumber": "string",

                "ChannelNumber": 0,

                "RegisterContentCode": "string",

                "PeriodOfAvailability": "string",

                "UnitOfMeasurement": "string",

                "EnergyFlowDirection": "string",

                "AccumulatorType": "string",

                "SwitchReadIndicator": false

              }

            ]

          }

        ]

      }

    ]

  },

  "Messages": [

    "string"

  ]

}


'Search' call – successful response sample

{

  "Address": {

    "PropertyNameOrDescription": "string",

    "PhysicalAddressStreet": "string",

    "PhysicalAddressSuburb": "string",

    "PhysicalAddressTown": "string",

    "PhysicalAddressRegion": "string",

    "PhysicalAddressPostCode": 0,

    "GPS_Easting": 0.11158078265068716,

    "GPS_Northing": 0.53635229909462989

  },

  "ICPIdentifier": "string",

  "ICPStatus": 0,

  "Messages": [

    "string"

  ]

Order by: Standard | Newest | Votes
Clive Gifford posted this 20 February 2020

The property name field can already be read.

Here are a few examples:

0000009476TE0DB WINDY FARMS KAITAIA
0000080993CPF7E MCKEE ENGR WORKSHOP
0000553257NR3D0 SECURITY GATES; UNMETERED
0000580475KEC30 GPS-2482330-5758289
0001701570WM8B8 WAIKATO TRAMPING CLUB
0006210642WM174 MRS J...S HOUSE       (trimmed here for privacy reasons)
0007110320RNB06 RESIDENTIAL-BR...     (trimmed here for privacy reasons)
0010399200EL1EA TAITOKOSCHOOLBDTRUSTEES
0030046153PC884 TARANAKI FISH & GAME COUNCIL
0089258600PC312 KAPUNI WELL SITES 1-7
0168568896LC8E0 METER ID AS AT 31/08/2010
0342894021LCEB6 MANGERE BRIDGE LIBRARY
0900086995PC776 HYDROLOGICAL MONITOR
1001162580UN12B NAVY BASE - RIFLE RANGE GATE
1099578323CN3B9 SYNLAIT SUPPLY NUMBER 2

Was the EA aware (before now) that this information could already be read via the ICP connection data API?

  • Liked by
  • msouness
Matthew Keir posted this 24 February 2020

Hi Clive,

Thanks for your response. The ICP connection data API hosted here has never released the 'Property Name' field.

The registry and the registry web-services –  which distributors, traders and MEPs all have access to –  provides access to the ‘Property Name’ field. To be clear, the registry web-services are not the same as the ICP connection data API. 

The API fields in the ICP connection data API were consulted on in 2015 and the API went live in 2016. No changes have been made to the API except to improve logging and increase the throttling to reduce the volume of concurrent calls from users.

Following the release of the original API, many users have requested the additional release of the ‘Property Name’ field. These requests include comments on this forum, the market performance post-implementation review, interviews held by the market design team, and more recently through the consultation on the ACCES quick wins enhancements

Based on the feedback above, the new version (version 2) of the API that is set to be released on 1 March 2020 will include the ‘Property Name’ field (released as ‘Property Name or Description’). As part of this process, distributors have all been asked to strip any private data from the ‘Property Name’ field in the registry prior to the 1 March release date. 

Some of the data you’ve posted is quite old – so your source is unrelated to the development and testing of version 2 of the ICP connection data API discussed in the post above. Some of these fields in the registry have not had the values you’ve posted since 2017.

Cheers,

Matthew 

Clive Gifford posted this 24 February 2020

Hi Matthew

I will take your message as a statement that nobody in the EA realised that the information could be read using only the ICP connection data API. Please correct me without delay if that is not correct.

All the information I displayed above was extracted from the registry, by myself, this year, and by only using the ICP connection data API - specifically the current version (as in place since 2016). I did not mean I somehow already had access to the v2 API.

As for your statement that some of the information I showed has not had the value shown since 2017, if you are referring (perhaps) to differences only in case, I displayed everything as upper case only because (and here's a little clue) the method I used to extract the information could not distinguish between upper and lower case.

Technically, I believe this puts the EA in breach of the Privacy Act, because of the various customer name information that was accessible.

Clive

Matthew Keir posted this 24 February 2020

Hi Clive,

Thanks for coming back again. We are investigating further and will contact you offline for further details.

Cheers,

Matthew

funcsol posted this 01 March 2020

"Migrating to the new version should be a simple change to the URL call to include (v2)"

The documentation at https://emi.portal.azure-api.net/docs/services/ hasnt been updated, & its not clear where in the url the v2 is added. Is this live? 

Phil Bishop posted this 01 March 2020

On Friday it was announced that there would be a delay of about two-weeks to the release of v2 of the API.

Phil

funcsol posted this 01 March 2020

Thanks for that! I managed to miss the announcement obviously. Notwithstanding, is there new documentation for v2?  

Phil Bishop posted this 02 March 2020

Regarding v2 documentation, it will pretty much look like the v1 documentation except for a few more examples, but it won't be available until the v2 API is released. In the meantime, the new fields that will be available in v2 are described in the first post of this discussion.

Clive Gifford posted this 12 March 2020

Will the PropertyName field continue to be searchable in the v2 API, as was possible before Friday 28th Feb, and as shown in the documentation for the API as it currently stands?

V1 API documentation

Matthew Keir posted this 17 March 2020

Hi Clive,

The aim of the search call was to enable someone to search an address in order to find out the corresponding ICP number. However, the ‘property name’ field in the registry has typically been used as a property description field rather than a genuine address field. The use of the field as a description is to help distinguish between multiple ICPs at the same address etc. Therefore, in v2 of the API, we are likely to keep things simple and only allow the current address to be searched, returning all ICPs at that address. The ‘property name’ field will be released as ‘property name or description’ in the result set for each ICP, as detailed in the original post above, pending any tweaks that might emerge following the review currently underway.

Cheers,

Matthew

RCCKHK posted this 26 March 2020

Is there any update on the release date of v2 of the API?

Phil Bishop posted this 01 April 2020

v2 of the API should be ready to go live next week. Hopefully we can settle on a precise date in the next day or two.

Phil

RCCKHK posted this 23 April 2020

Good Afternoon :-) Just wanted to touch base to see if you've managed to settle on a precise date for the release yet? 

funcsol posted this 25 May 2020

Accounting for global pandemics & recent earthquakes... maybe this can be renamed ACCES Average Velocity Wins. ;-)

Matthew Keir posted this 25 May 2020

Yes slower than we'd planned - v2 is now live. See https://forum.emi.ea.govt.nz/thread/new-emi-api-release-version-2-of-the-icp-connection-data-api-now-available/

This discussion is locked, please start a new discussion.