We are in the midst of modifying the GDX creation process as well as the process for publishing both final pricing GDX and SPD case files to EMI datasets. The following locations are where case files and GDX files are being published via the new process:
- https://www.emi.ea.govt.nz/Wholesale/Datasets/FinalPricing/CaseFiles_NEW
- https://www.emi.ea.govt.nz/Wholesale/Datasets/FinalPricing/GDX_NEW
There is no need for users of either SPD case files or GDX files to do anything just yet.
Once we have finished implementing and testing the new processes, we will announce the date on which the current processes will be terminated. Plenty of time will be allowed for users to modify scripts and automated processes used to obtain GDX and case files. But if this is going to affect you, you might want to start thinking about it now.
On the yet-to-be announced day that we turn off the current process, two things will happen:
- the contents of ../FinalPricing/CaseFiles will be moved into ../FinalPricing/CaseFiles_NEW, ../FinalPricing/CaseFiles will be deleted, and ../FinalPricing/CaseFiles_NEW will be renamed ../FinalPricing/CaseFiles
- the contents of ../FinalPricing/GDX will be moved into ../FinalPricing/GDX_NEW, ../FinalPricing/GDX will be deleted, and ../FinalPricing/GDX_NEW will be renamed ../FinalPricing/GDX.
The GDX file naming convention has been modified relative to what's being done at present. As can be seen from time to time in ../FinalPricing/GDX_NEW/2020, newly published GDX files will initially have a name something like: FP_20201021_201112020101100806_20201022074928.gdx. This file name is made up of three components:
- the trading date to which the GDX file pertains, e.g. 20201021 in the example above
- the SPD case ID from which the GDX file was created
- the run date and time at which the case file was created, e.g. 07 hrs, 49 mins and 28 secs on 22 Oct 2020 in the example above.
Once we learn the price type associated with the SPD case file and the commensurate GDX file, i.e. provisional or interim, the _P or _I suffix will be added. Eventually prices will go final and the _F suffix will be added. At that time, the GDX file will be named as described here and all other GDX files pertaining to that day will be deleted, i.e. once final prices have been declared and published, there will only be a single GDX file available for that day from EMI datasets.
All of the steps in the new process are driven by event triggers in the Azure environment so in addition to a general reliability improvement, we expect the new process to reduce the time it takes to make GDX and case files available.
