The Great File Migration Is Underway - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Cloud // Cloud Storage
08:06 AM
Connect Directly

The Great File Migration Is Underway

As the amount of data in files grows, more and more will find their way in a great migration to cloud storage.

IT Hiring, Budgets In 2015: 7 Telling Stats
IT Hiring, Budgets In 2015: 7 Telling Stats
(Click image for larger view and slideshow.)

Two powerful trends are conspiring to force files out of the data center. Relentless data growth combined with the desire for high-performance flash arrays is making IT organizations rethink their file storage strategies. Files are being driven out of the data center, and they are heading to the cloud. As a result, the data center of today is faster, more efficient, and thinner than ever.

Flash storage arrays provide an ideal foundation for the IO-hungry workloads of virtual machine hypervisors, but they don’t solve the problem of files, which is one of scale and cost. A typical organization experiences 30% or more year-over-year compound growth in their file data. The problem is particularly acute in file-heavy industries such as design, engineering, media, and medical.

Todd Thomas, CIO at the Austin Radiological Association, illustrates this point when he shows their projected file growth. ARA is one of the largest medical imaging centers in the country, performing over a million exams annually. The chart below shows the file footprint just for mammography imaging alone. Today mammograms account for about 7 terabytes of storage, but that footprint is expected to double by the end of the year. Not only are there more files, but the files are getting much larger as imaging moves from 2D to 3D.

Although flash is becoming more affordable every year, it is no match for the rapid expansion in the file footprint. The widespread adoption of server virtualization drove the proliferation of SAN. This has been quickly followed by a demand for higher and higher performance from SAN. The data center wants to be all virtual. Virtualization improves the efficiency of the data center by making workloads more fluid. Electrical power can be directed to exactly where it is needed instead of having to over-spec hardware, only to find out later that even bigger hardware is required. As more server-class workloads break away from the underlying hardware through virtualization, disk IO becomes the bottleneck preventing hypervisors from handling heavier workloads.

[Want to learn more about how files are central to future storage systems? See NAS Is Sexy Again.]

Pain in the IO path has caused a slew of flash vendors to appear out of nowhere, which in just a few years have conquered the once unassailable mission-critical application workloads belonging to the Goliaths of the storage industry. Vendors like Pure Storage, SolidFire, Tegile, and Nimble Storage have pioneered the use of data de-duplication in flash in order to make the technology more affordable. At last, the all-virtual data center seems within reach. The last obstacle is files.

A successful transition to an all-virtual data center is predicated on being able to afford the premium that flash storage commands. Enlightened IT organizations are taking a two-pronged approach that consists of moving all of their server workloads to flash while pushing files to cloud-based NAS. The cloud has become such an appealing option, because it is essentially a giant data replication machine operated at a global scale by the likes of Amazon, Apple, Google, and Microsoft. As a result, the cloud makes NAS bottom-less, backup-less and much more affordable than anything built to run exclusively inside the data center.

This new way to store files uses synchronization to periodically update a master copy of not only every file, but also of the file version in these massive stable backends. Cloud NAS combines the stability and scale of the cloud into an on-premises appliance that fits neatly into the expectations for performance that exist in the modern data center.

This two-pronged approach to storage is no different from what has been happening to consumer laptops. Almost every new laptop today worth having ships with flash storage. Flash is faster, more reliable and less power hungry than a spinning hard drive, but it is also smaller and more expensive. But that’s OK, because, today there are also fewer files in a laptop.

Think of all that music being streamed instead of being held hostage inside your laptop. Apple recently introduced Photos, a whole new application for storing pictures. The exciting new feature: your pictures are actually stored in the Apple cloud, not your laptop. By removing photos from laptops, Apple hopes to sell faster, thinner, and more affordable laptops. Apple will also be making it easier and less risky for anyone to dump that old laptop because there is no real data in it. The master photo library stays in the cloud.

The great file migration is underway. The combined effect of continued file data expansion with a shift towards high performance arrays has caused organizations to adopt cloud NAS as a way to handle their file capacity needs. Cloud NAS has the additional benefit of storing the files outside the data center, free from the limitations of traditional storage. This approach not only simplifies hardware refreshes, but also ensures that the files are safe no matter what happens at the data center.

The problem with files is one of scale. The cloud is practically infinite. The great file migration to the cloud is inevitable.

[Did you miss any of the InformationWeek Conference in Las Vegas last month? Don't worry: We have you covered. Check out what our speakers had to say and see tweets from the show. Let's keep the conversation going.]

Andres Rodriguez is CEO of Nasuni, a supplier of enterprise storage using on-premises hardware and cloud services, accessed via an appliance. He previously co-founded Archivas, a company that developed an enterprise-class cloud storage system and was acquired by Hitachi Data ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
Charlie Babcock,
User Rank: Author
5/22/2015 | 6:43:19 PM
A global file systemj with headquarters in the cloud
In his Sept. 25, Andres talked about the need for a single global file system based outside the enterprise data center. It would  make use of the cloud's characteristic of keeping three copies of the data in order to be able to survive a hardware failure. If one or more copies is kept geographically separate as well as in a separate availability zone, cloud storage lends itself to strong data survivability characteristics through this cloud practice, without layering on top its own "make a copy: replications. In corporated cloud architecture as a given and available for global file system creation, accessible by all. I think we're headed i that direction.
Charlie Babcock
Charlie Babcock,
User Rank: Author
5/21/2015 | 8:46:47 PM
What makes file storage in the cloud inevitable?
Is Andres right? Will files inevitably find their way into some form of cloud storage? Or will the capacities of on-premises storage systems keep up with the wild growth of information? And if you have an on-premises solution, will it have the automated replication and second site storage of the cloud?
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

10 Things Your Artificial Intelligence Initiative Needs to Succeed
Lisa Morgan, Freelance Writer,  4/20/2021
Tech Spending Climbs as Digital Business Initiatives Grow
Jessica Davis, Senior Editor, Enterprise Apps,  4/22/2021
Optimizing the CIO and CFO Relationship
Mary E. Shacklett, Technology commentator and President of Transworld Data,  4/13/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Flash Poll