Cloudy, with a chance of performance

The most significant challenges of our time in research computing are almost always those of scalability. When we build a super computer, it's quickly found to be too small and inadequate to cope with demand. When we build a high capacity storage area network, we find that users fill it and then ask for more, as if it were a gas expanding into an open space. 

With the advent of private, public and hybrid clouds being accessible to the masses, the notion of almost unlimited performance at a small cost is in theory achievable. How does this translate to seamless end user experiences and workflows being vastly improved, much less research outcomes being markedly better? 

The reality is often different from this grand vision of seamless performance and on-demand scale. 

In this presentation, Jake and Jason will discuss their experiences as the men behind the architecture and delivery of the Queensland RDSI and research cloud nodes, including the performance bottlenecks, considerations, technology decisions and the complex problems that arise when an architect sits down and really thinks about a way to satisfy the research requirements of as many researchers as possible, rather than simply building big iron for the sake of building big iron.

Easier said than done, this presentation is a journey of failures, more failures, epic failures, hard analysis, benchmarking and finally successes in and around storage and cloud architecture. There might be some strategy to it all, too!