Users of the frequency keeping dispatched offers data occasionally notify us of anomalies in that data.
We are aware that duplicates and other irregularities sometimes appear in the daily XML files (there is no problem with the monthly CSV files). The problem stems from the way the system operator transfers the daily data from its market systems to its non-market systems, and then to the Authority.
Once published on EMI, the daily XML files are made available for 40 days and are then removed. The Authority publishes these files as soon as they are received and without any modification. While it is easy enough to identify anomalous data, we don’t always have the required information to resolve the problems in a timely manner. Users of the data have previously indicated to us that they would prefer to have access to the data sooner, even if it occasionally contains irregularities, rather than wait several days to be assured it is accurate. That is why we make no attempt to check and correct the daily files.
However, the data that goes into the monthly CSV files is provided to the Authority via a different process than the daily XML files. It is not as timely but it is reliable.
