9 FAQs

Will AEMO test the PDR applications both old and new versions of Java?

Moving forward, the idea is to test these updates internally on the latest supported versions which is OpenJDK21. You can run application on any version JRE11 and above.

Please make sure to do your own validation in a non-production environment first.

Which Windows Server version will the new data interchange support?

AEMO Australian Energy Market Operator does not certify against any particular operating system. The JRE sits above the operating environment and different participants use different systems and AEMO cannot take on all the different possibilities.

Is there going to be another release ahead of IDX, or is this the final build?

Yes, there will be another IDX release. The message exchange and identity patterns are still under discussion in AEMO. The approach is to implement functionality that is compatible with the final target state build for IDX.

Are the translators in Batcher to support workloads like loading into data lake solutions like data bricks or fabric?

You can use the translators for a variety of purposes including loading data into data lake solutions, but their use is entirely up to the participant. AEMO provides most data from the participant file server in the AEMO csv Comma Separated Values. A file format for data using commas as delimiters. format for efficient data replication and loading into relational tables. If you want to convert to JSON JavaScript Object Notification or another format, the translation facilities are available to support this, and participants can configure them based on their use case.

Do participants need to upgrade to the new version of Data Interchange applications and what is the impact if they do not upgrade?

AEMO recommends upgrading to the latest version of Data Interchange A set of cooperating applications used to replicate data between AEMO's energy market systems and a participant's DBMS conforming to the MMS Data Model.. However, there is no immediate impact if you do not upgrade. Older versions may continue to function, but if an issue arises, support for the older versions is limited. Participants may have to upgrade to a supported version to resolve certain problems. Additionally, some new functionalities are only available in the latest versions.

If participants are on pdrBatcher (7.4.2), will they need two incremental upgrades, from 7.4.2 → 7.5.0 → 7.6.0?

For the pdrBatcher Software responsible for transferring files to and from AEMO's participant file server to the participant's local Data Interchange folders. It can be installed in any environment supporting Java., a direct upgrade to 7.6 is possible if you are running a vanilla configuration. You can install v7.6 alongside 7.4.2 and choose when to cut over. However, for the pdrLoader Software responsible for loading files to and from participant's local Data Interchange folders to the participant's DBMS. It can be installed in any environment supporting Java., incremental upgrades are required due to database changes in the PDR management tables affecting both structure and data. AEMO does not recommend skipping versions in the pdrLoader as these database upgrades must be applied sequentially.

What is the impact of not updating on time?

PDR apps are different from data model tables. If you don't upgrade PDR apps, the data is still received, and there is no impact. You can analyse your system and upgrade in your own time. However, the support for these applications is for versions N & N-1. If there is a support issue and your version is N-2, upgrading to a supported version is required.

Is there a link for pdrBatcher and pdrLoader documentation?

All PDR application documentation is available on the Data Interchange Help.

What does it mean by custom APIs?

We consider them internal APIs. The custom APIs mentioned are for the APIs that PDR monitor uses. The PDR monitor supports these internal APIs, ensuring that the data interchange and monitoring functionalities are maintained.

How does the process work for polling the BLOB storage for the loader?

The process involves the loader checking the BLOB storage at regular intervals to see if there are any new files to process. Once it detects new files, it picks them up and processes them accordingly. This ensures that the data is continuously updated and available for further analysis.

Could you comment on the mentioning of sparse data for the pdrLoader new release?

The new release of the pdrLoader includes a feature for handling sparse data. This feature ensures that the system can efficiently manage and process data even when it is sparse, improving overall performance and reliability. The implementation optimises data storage and retrieval processes to handle sparse datasets effectively.

Can pdrMonitor manage multiple versions of pdrLoader/Batcher?

Yes, it can. The new version of pdrMonitor still supports older versions and unencrypted connections to application instances.

Is the congestion monitor designed to address the issue of next day bid files creating lag spikes until they are imported properly?

Yes, the congestion monitor addresses this issue. It identifies and manages congestion by monitoring data transfer spikes and ensuring that bid files are imported efficiently. This tool aims to minimise lag and improve overall data processing performance.

Can you add a label or rename the Widget?

The naming convention for a Widget is attained from Data Interchange Monitor->Configurations->Connections under Service Label. The option to have a label added or renamed is not currently available, however AEMO considers this as a feature request.

Any plans to add LDAP integration for pdrMonitor user logins?

AEMO has an implementation of LDAP logins, where it isn’t single sign-on but uses the same password. The default installation of pdrMonitor uses a local user data store but can be changed to use LDAP with configurations.