HPC

The High Performance Computing (HPC) Project targets a priority in the Australian Astronomy Decadal Plan (2016 – 2025) of world-class HPC and software capability for large theoretical simulations, and resources to enable processing and delivery of large data sets from these facilities.

The GADI supercomputer located at NCI. Credit: NCI.
The SkyMapper telescopes finishes its final observations for the night under moonlight at Siding Spring Observatory, while dawn begins to break on the horizon. Credit: Angel Lopez Sanchez.
CASDA allows astronomers around the world to access data from ASKAP. Credit: CSIRO.
The new Square Kilometre Array Observatory (SKAO), a next-generation global radio astronomy facility being built in Western Australia, will produce multiple petabytes of data in a single night. Credit: SKAO.
The OzSTAR supercomputer, located at Swinburne University of Technology. Credit: Carl Knox, OzGrav.

AAL is addresses this priority through the purchase of high performance and cloud computing resources in partnership with the National Computational Infrastructure, CSIRO, and Swinburne University of Technology.

Gadi

AAL purchases service units on Gadi, Australia’s most powerful CPU-based research supercomputer, based at the National Computational Infrastructure (NCI). This time is available to the Australian astronomical community via a competitive review process, overseen by the AAL Supercomputer Time Allocation Committee (ASTAC).

Researchers affiliated with an institute in Australia are eligible to apply with the time awarded to projects running large-scale parallel computations.

Australian Optical Data Repository (ODR)

AAL has partnered with NCI to establish the Australian Astronomy Optical Data Repository (ODR).

The focus for this project is for NCI to provide the data repository operation including data management, data services and software and data release processes, which underpin:

  • The contemporary Anglo-Australian Telescope (AAT) datasets and selected historical datasets from the AAT archives; and
  • The SkyMapper telescope datasets including the Southern Sky Survey, and selected legacy optical data collections as time and priority allows, such as MACHO and surveys from the ANU 2.3m telescope.
CSIRO ASKAP Science Data Archive (CASDA)

CASDA is a collaboration between CSIRO and the Pawsey Supercomputing Centre to build, store and archive ASKAP data and to make the data accessible to astronomers around the world. CASDA stores science-ready data products produced by the CSIRO custom-built software package ASKAPsoft. 

CASDA has been in development since 2013, with the first release in late 2015. Several new releases of CASDA with additional enhancements have been made in the last few years. More information on CASDA can be found on their website.

Supporting YANDAsoft on external HPC platforms

This project commenced in 2019 with the aim to provide support to the wider ASKAP community to install and use YANDAsoft (formerly ASKAPsoft, the astronomical calibration and imaging software pipeline for ASKAP) on HPC platforms outside of the Pawsey Supercomputing Centre, therefore maximising the quality and efficiency of the science from the ASKAP telescope.

This funding enabled CSIRO to support installation of YANDAsoft on other HPC platforms such as Niagara in Canada, OzStar in Swinburne, INAF in Italy, Nimbus at Pawsey, DUG in Perth, and a cluster at ANU.

AAL support for YANDAsoft concluded at the end of 2021 but YANDAsoft development continues as part of the Australian SKA Regional Centre Design Study Program.

OpenStack

Swinburne University of Technology offered readily available access to 1,000 OpenStack Virtual Machines (VM) on the dedicated Swinburne cell within the NeCTAR Cloud network to the national astronomy community. This project facilitated uptake of existing NeCTAR resources by the astronomy community by developing a customised astronomy interface, conducting user tutorials, and identifying small job use cases on OzSTAR that were suitable for transfer from HPC to the VM infrastructure.

AAL support for this project has now concluded but Swinburne continues to make the project resources available for astronomy usage, including access to the OpenStack VMs, access to the sstar compute nodes, availability of user documentation, online training materials and direct assistance if required.