By Nuwanthika Fernando
[email protected]
In 2019, a team of software engineers from Astralis-AAO (Macquarie University) and Astralis-AITC (ANU) took over the maintenance and upkeep of 19 data processing pipelines used by the VLT, as a part of the ESO Pipelines Project. Currently in its 3rd successful year, this project between ESO and the Astralis Instrumentation Consortium now supports 22 software components.
VLT software pipelines are used to support astronomers with the calibration, reduction and analysis of the various raw data products of the VLT. They are constantly improved, with the changes driven by requests from Instrument Scientists and various user groups. Each pipeline is tailored to its instrument, and therefore the first year of the project involved the Australian team gaining knowledge of the extensive software infrastructure that supports these complex instruments. In 2020, the Australian team took over the official annual releases for Paranal in April, followed by a Public Software Release in June for the assigned pipelines.
This challenging task was accomplished with guidance from ESO’s Pipeline Systems Group, local technical support from the Astralis-MQ IT group, countless meetings spanning three continents, and several in-person visits to the ESO Headquarters in Garching, Germany. Despite the steep learning curve, demanding testing standards, and travel restrictions due to a global pandemic, in March 2020 the team delivered 11 upgraded pipelines ready for installation at the VLT. However, unlike the pipeline versions that are used in uniform software environments at the telescope, public releases need to run in the myriad of platforms used by astronomers in general. The team’s efforts to achieve all these requirements in less than two months resulted in the successful delivery of 19 pipelines to the public.
One new and exciting component of the Project is the effort to create a Python interface for the Common Pipeline Libraries (CPL) used by all VLT instrument pipelines. This will allow users to call currently available C/ C++ based pipeline ‘recipes’, or data-reduction tasks, from Python, and eventually create new recipes using CPL features, for current and upcoming ESO instruments.
Having already undergone multiple successful reviews and a contract extension, ESO, the Astralis-AAO and Astralis-AITC teams are already looking towards new challenges for the next two years, and the ongoing success of the ESO Pipelines Project is sure to set the stage for more fruitful collaborations in the future.