Riding the Super Fast Storage Tsunami - Solid-State Disk Considerations in Next-Gen Storage Infrastructure
The catch cry of every technology journalist and storage vendor currently revolves around the mass data growth or ‘data explosion’ the world is facing. Structured and unstructured data is being generated at a rate that organisations are finding it hard to come to grips with. With the rise of next generation full genome sequencers, high resolution sensory networks, super dense MRI devices and ultra resolution CCD’s, the scientific and research sector is placed in a difficult position, to make decisions about the way data should be captured, stored, curated and then accessed.
Within this access, there are expectations from users that are growing (arguably) unrealistically beyond where current generations of storage infrastructure are capable of servicing, such is the nature of the enormity of the data being requested, compared with the amount of storage and throughput capability currently available.
With this issue of throughput (and latency) in mind, NAND-flash-based solid-state disk (SSD) media is becoming a prevalent killer application, with adoption in the high-end research sector and corporate enterprise markets growing steadily.
Of particular interest to our organisation were the long-term performance characteristics of SSD storage devices in the face of factors such as capacity utilisation, wear leveling and cell attrition. On this basis, we set out to understand how several different SSD based products behaved given scientific research data workloads (which are typically qualified as mixed, unpredictable and unstructured, compared to a generic enterprise workload).
The impetus and motivations for this research were three-fold.
- To try and understand whether or not the vendors, manufacturers and brands behind these technologies are describing an accurate picture of performance to us, as demanding and mission critical users.
- To gain a better insight into the technology, as a means to improve it for future users and implementations.
- There is a defined gap in the body of knowledge that exists for such implementations in our observations, which suggests many organisations may take the arguably unqualified advice from vendors, suggesting that SSD is the ‘silver bullet’ and solve-all style solution, which may not be the case for specific use-cases and problem domains.
This research (and subsequent presentation) demonstrates and explains our findings through scientific method and benchmarking protocols. We then go on to explain the reasons behind the SSD performance patterns in our findings, dispel some of the common myths about the technology, and highlight some of the implementation pitfalls that organisations often fall into, such that, future users and organisations will have more understanding and knowledge, to make a more strongly informed decisions, for their future purchases. Within this, we offer suggestions and possible solutions to SSD utilisation that can ultimately make the current generation of hardware more usable and safe for targeted applications.
|
Jake Carroll's Biography |
