HPC

The High Performance Computing (HPC) Project targets a priority in the Australian Astronomy Decadal Plan (2016 – 2025) of world-class high-performance computing (HPC) and software capability for large theoretical simulations, and resources to enable processing and delivery of large data sets from these facilities.

The GADI supercomputer located at NCI. Credit: NCI.
The SkyMapper telescopes finishes its final observations for the night under moonlight at Siding Spring Observatory, while dawn begins to break on the horizon. Credit: Angel Lopez Sanchez.
CASDA allows astronomers around the world to access data from ASKAP. Credit: CSIRO.
The OzSTAR supercomputer, located at Swinburne University of Technology. Credit: Carl Knox, OzGrav.

AAL aims to address this priority through the purchase of high performance and cloud computing resources in partnership with the National Computational Infrastructure (Gadi and ODR), Swinburne University of Technology (OpenStack), and CSIRO (CASDA).

Gadi

AAL purchases service units (SU) on Gadi, the new Fujitsu system at the National Computational Infrastructure (NCI). This time is available to the Australian astronomical community via a competitive review process, overseen by the AAL Supercomputer Time Allocation Committee (ASTAC).

Researchers affiliated with an institute in Australia were eligible to apply with the time awarded to projects running large-scale parallel computations.

Australian Optical Data Repository (ODR)

AAL has partnered with NCI to establish the Australian Astronomy Optical Data Repository (ODR).

The focus for this project is that NCI will provide the data repository operation including data management, data services and software and data release processes, which underpin:

  • the contemporary Anglo-Australian Telescope (AAT) datasets and selected historical datasets from the AAT archives
  • the SkyMapper telescope datasets including the Southern Sky Survey, and selected legacy optical data collections as time and priority allows, such as MACHO and surveys from the ANU 2.3m telescope.
OpenStack

The Swinburne University of Technology offers readily available access to 1,000 OpenStack Virtual Machines (VMs) on the dedicated Swinburne cell within the NeCTAR Cloud network to the national astronomy community. This project will facilitate uptake of existing NeCTAR resources by the astronomy community by developing a customised astronomy interface, conducting user tutorials, and identifying small job use cases on OzSTAR that are suitable for transfer from HPC to the VM infrastructure. These activities will ensure that the resources are well utilised and will create more space on the HPC system for larger-scale jobs.

CSIRO ASKAP Science Data Archive (CASDA)

CASDA is a collaboration between CSIRO and the Pawsey Supercomputing Centre to build, store and archive ASKAP data and to make the data accessible to astronomers around the world. CASDA stores science-ready data products produced by the CSIRO custom-built software package ASKAPsoft. 

CASDA has been in development since 2013, with the first release in late 2015. Several new releases of CASDA with additional enhancements have been made in the last few years. More information on CASDA can be found on their website.

Supporting YANDAsoft on external HPC platforms

This project is to provide support to the wider ASKAP community to install and use YANDAsoft (the astronomical calibration and imaging software) in HPC platforms outside of the Pawsey Supercomputing Centre. The intent of this undertaking was to maximise the quality and efficiency of the science from the ASKAP telescope, and in late 2019, ADACS successfully deployed YANDASoft onto the OzSTAR computing cluster at Swinburne University of Technology.

The scope of work is listed below:

  • Continuing supporting the installation of YANDAsoft at several external HPC platforms including Swinburne cluster, ANU HPC platform and Research Cloud Services in Australia, INAF (Italy) and CICADA (Canada).
  • Maintain YANDAsoft user and developer documentation.
  • Continuing improvements to the YANDAsoft development and deployment software infrastructure, including improvements in the continuous delivery pipeline and testing coverage, in particular supporting containers-type of deployment such as Singularity.