The AGRF has deployed Cloudian’s S3-compatible object storage solution, enabling AGRF to manage and store its rapidly growing volumes of genome sequencing data more effectively, Cloudian announces.
The AGRF is said to empower world-class genomic science in the biomedical, clinical, agricultural and environmental sectors, with over 15,000 clients worldwide. The organisation has been at the centre of COVID-19 research, from assisting researchers with vaccine development to detecting the virus in wastewater. AGRF is also distinguishing itself by sequencing genomes of Australian threatened species to aid conservation efforts.
In an average week, AGRF processes at least 10 terabytes (TB) of data—equivalent to one year’s worth of data from the Hubble Space Telescope. Because this data drives important research, AGRF needs a storage system that enables it to easily manage, protect, and access the data at the lowest possible cost.
“We are a data organisation in every sense of the word,” says AGRF ICT manager Douglas Morrison. “Mapping the genome of one patient involves 100-125 gigabases, and easily three times that if we are analysing specific tumour samples. Multiply that by 1,500 patients or so in just one typical study, the data adds up fast.”
“Some of our clinical clients require urgent turnaround times for decisions on how to treat a patient—particularly in the case of cancer screening. We are working on a project now where we are aiming to significantly decrease our turnaround times to enable faster decisions. That accelerated process is possible because of how quickly we can handle all the data,” he explains.
AGRF had been using a tape backup system which involved 16 hours of manually swapping tapes in and out every week. This process was a huge burden for its small IT team and had begun to impact operations.
“Because we had to spend so much time on backups, we were not always available to assist staff if they had an issue arise,” laments Morrison. “In addition, if we needed to retrieve backup data for a project, they had to wait for us to locate the right backup tape and restore it.”
As its data volumes continued to grow, AGRF realised it needed a better solution. After evaluating alternatives including tape systems and public cloud, the facility chose the Cloudian HyperStore solution for its scalability, ease of management and cost-effectiveness.
“We liked that Cloudian provided the scalable capacity to meet our future growth needs without the hassle and time required in managing tapes and accessing data,” observes Morrison. “In addition, the Cloudian HyperStore solution was half the cost of public cloud storage, a huge consideration for AGRF as a not-for-profit organisation.”
AGRF deployed HyperStore appliances with a total capacity of 1.8 petabytes, using Cloudian’s HyperFile software to move data to HyperStore. AGRF claims the adoption of the Cloudian solution has brought huge benefits.
Cloudian says it has also enabled AGRF to increase its footprint in fast-growing applied markets and commercial diagnostic industries. In addition, the organisation will leverage HyperStore in an upcoming project to map the genome of the infamous Australian box jellyfish, helping researchers to develop improved antivenoms and possibly therapeutic adjuncts.
“The importance of genome sequencing and mapping has become even more apparent over the past year in the effort to combat the pandemic,” concludes Cloudian regional director, A/NZ and Ocenia James Wright. “This industry is driving medical advancements all over the world and is key to beating the virus.
