EIEP13 consumption file examples

  • 2.2K Views
  • Last post 25 January 2024
Matthew Keir posted this 27 June 2016

EIEP13 files are used to request consumer consumption data in a standard format.

The easiest way to see some examples of the data returned is to start requesting them. However, I’ve managed to track down a real example of the 13A and 13B formats to help give people a flavour of what they’ll get – you’ll see that I’ve anonymised them by removing the ICP and meter serial number from the files. I’ve attached these to this discussion.

More information on the EIEP13A, B and C formats is available here.

Information on the procedures to request consumption data using EIEP13C is available here.

I’ll update this post when I have an example of an EIEP13C request file.

 

[Edit: originally posted sample file for EIEP13A replaced with modified version on 2/08/2023 by Guy Ross.]

Attached files

  • Liked by
  • energie
  • gardner
Order by: Standard | Newest | Votes
Clive Gifford posted this 27 July 2023

The published sample for the EIEP13A file given above is not compliant with the specification.

The date/time fields can be easily seen to be using a mixture of NZST and NZDT but the "NZDT adjustment" field is blank for all records through the file.

The fact that EA is publishing a non-compliant "sample" is disappointing to say the least. I've seen another EIEP13A file in the real world that shows the same fault. Hello Contact.

Without NZDT or NZST being specified at any point in the file then it is impossible to distinguish between two records on the dates that daylight saving ends.

In this sample of a NON COMPLIANT file the first examples of the problem that can be easily seen  are in the record numbers 8886 through 8889. There you see two records for 2 am to 2:30 am and another two for 2:30 am to 3:00 am, on 5 April 2015.

DET   XXXXXXXXXXXXXXX 0   XXXXXXXXX X 7304 24 05/04/2015 02:00:01 05/04/2015 02:30:00 RD 0.1 0 
DET   XXXXXXXXXXXXXXX 0   XXXXXXXXX X 7304 24 05/04/2015 02:30:01 05/04/2015 03:00:00 RD 0.1 0 
DET   XXXXXXXXXXXXXXX 0   XXXXXXXXX X 7304 24 05/04/2015 02:00:01 05/04/2015 02:30:00 RD 0.09 0 
DET   XXXXXXXXXXXXXXX 0   XXXXXXXXX X 7304 24 05/04/2015 02:30:01 05/04/2015 03:00:00 RD 0.19 0


Order of records in the EIEP13A file specification is not forced. This means you can't tell which is which out of pairs that have the same date/time details.

Of course, after noticing this we could *ASSUME* the the preceding records are NZDT, and NZST follows until the next period of NZDT begins but having to decode each file in that way is nonsense. We could also *ASSUME* the records are in chronological order and so *GUESS* which of each pair of conflicting records should be first.

If for some reason the EA truly believe this file is a good example of compliant file (with respect to date/time information for the half hours and the "NZDT adjustment") then the specification needs to be changed to fully explain how that is true.

The EIEP13A format is meant to be for end use of consumers. I don't believe there is a way to interpret such as file as compliant although many may naively look at it and assume they know what it means and so "be happy".

The format and specification has to be such that ordinary mortals can read the document and have a good chance of correctly interpreting it.

I'll take this opportunity to also say that allowing for "BLANK" in the "NZDT adjustment" field is an invitation for confusion and error. Especially as having BLANK means the same as "NZDT Adjusted" (i.e. the date/times are UTC+13). What is the logic behind that anyway?

In addition, in the real world clock half hour intervals start with a time such as 10:00:00 - not 10:00:01 as the specification seems to require. So the specification appears to have been written by someone who believe all times should be specified by the end point of some interval. Such stupidity means code written to generate the format or verify it when reading has to jump through some equally stupid and unnecessary nonsense.

I wonder how many consumers have tried to get/use EIEP13A files and then run across retailers that pretend or actually do know nothing about it, and then perhaps proceed to generate non-compliant files in all kinds of different ways?

The EA needs to do much better. Provide some short samples showing the possible correct ways handle different types of NZST/NZDT combinations especially across boundaries. A specification that simply required every record to explicitly state NZST or NZDT as appropriate makes it much simpler and clearer, at least for half hour data.

Consider this to be a request for change and of course rapid replacement of the non compliant sample file.

  • Liked by
  • David L Hingston
Guy Ross posted this 01 August 2023

Hi Clive,

Thanks for that. I believe in both cases the sample is compliant. I also agree that yes, the specification is sometimes lacking or counterintuitive, and could be improved. 

There appears to be two issues here.

1. Identifying daylight savings, and

2. Identifying the start/end of a period. 

For the first:

This is consistent with other longstanding interchange formats. 

The specification refers to the code: 

Refer to clause 15.36 of Part 15 of the Code. If information is NZDT adjusted, the field may be left BLANK, otherwise if it is not adjusted, ‘NZST’ must be used.

Which says:

(3) A daylight savings adjustment must be made by using the “trading period run on technique”, which requires that daylight saving adjustment periods are allocated as consecutive trading periods within the relevant day, in the sequence that they occur.
(4) If no adjustment is made in accordance with subclause (3) to information exchanged between reconciliation participants that contains trading period specific data, the code “NZST” must be used within the data transfer file.

My plain English interpretation of this, is that the default option is to use “clock” time (ie NZST in winter and NZDT for summer). Listed, in order, when going into summer time.  In this case, the time zone may optionally be left blank. 

Otherwise, if not using NZDT during summer time, then it must be identified explicitly as “NZST”

You’re right though, it certainly wouldn’t hurt to make the NZST/NZDT values mandatory, and we’ll look at encouraging that the next time the SDFG gets together.

 

For the second:

Yes. Most systems would agree that a period starts at hh:00:00, although that does just shift the interpretation issue to the other end of the period. (Is it hh:30:00? hh:29:59?  hh:29:59.999?).
From the specification:

5.1 If the interval of a consumption record is less than one whole day, the Time part of the DateTime formatted value must reflect the appropriate hours, minutes and seconds of the record (eg a half hour trading period record could have a start date/time of “01/03/2016 00:30:01” and an end date/time of “01/03/2016 01:00:00”). 

There’s elements of common sense, and, standard datetime interpretation issues at play here. Regardless of interpretation, 30 minutes still equals 30 minutes.

Again, some clearer standards, and references to things like inclusive start/exclusive end times will be encouraged next time the EIEP13 formats go out for consultation.

 

I’ve edited the sample to be clearer.

David L Hingston posted this 25 January 2024

I agree with Clive Gifford about non compliance.

The source version is 1.1 while the current version is now 1.4.

Perhaps Clive was in his own way alluding to the following as well?

Column F (when imported into spreadsheet form) rows 2 down and onwards are noted to be in the form of XXXXXXXXX

I am assuming Column F is the field "Metering component serial number" defined as

"Mandatory for a metering component. Identifies the metering component for installations that have multiple metering components.
For unmetered load “UNM”

as per protocols 1.1 to the current 1.4. Now it is defined as 30 chars (1.1 was 15) and compulsory.

Would it be more correct to be shown in the form of XXXXXXXXX:Z where a colon is a colon(!) and "Z" represents the particular register integer the data applies to - to cover the meters with more than 1 register, i.e. "multiple metering components"?

Comment and if I am correct in any respect review and correction of the example file is requested.