How can the research cloud support research communities?

Bernard Meade, Steven Manos, Richard Sinnott, Christopher Fluke, Dirk van der Knijff and Andy Tseng

Modern research is facing a new challenge: Big Data. Such is the wealth of information captured in these big datasets that for disciplines such as astronomy, more research publications are based on archived data than newly acquired data. The “data deluge”, generated by the next generation of scientific instruments, is growing far quicker than the capabilities of typical network infrastructure and computing resources. Desktop computers not only lack the capacity or processing power to deal with so much data, they lack the bandwidth necessary to transfer the files in a reasonable timeframe. Remote storage and computation is the only practical solution. While High Performance Computing (HPC) still plays a major role in this area, cloud computing offers several benefits over the traditional HPC approach to this challenge. It is not a case or either HPC or cloud computing – both a necessary for future research. 

In this presentation we will discuss how the NeCTAR Research Cloud can be employed by Australian researchers to meet the challenge of Big Data and enable and empower research communities. Because the value of the data goes beyond the initial collection motivation, further research based on a dataset or collection of sets is brought about by community awareness. This potential for reuse of data for entirely new research is a key ingredient to justifying expenditure on high-end resources, rather than myriad low-end resources. 

In disciplines such as astronomy or particle physics, new instruments produce vast datasets that are not able to be easily transferred from their storage sites. Working with these datasets is typically managed via Virtual Laboratories (VL), such as the All Sky Virtual Observatory or the Marine Virtual Laboratory. These and other VLs aim to bring collections of tools together to enable the research communities to share the resources, and to enable greater collaboration.