Supercomputers Unleash Big Data's Power - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IT Life
07:06 AM
Lisa Morgan
Lisa Morgan
Connect Directly

Supercomputers Unleash Big Data's Power

Here are six reasons why established companies, and even startups, are using supercomputing resources, and why your IT organization may want to consider such options to meet your big data and business analytics needs.
1 of 7

(Image: sputnikos/iStockphoto)

(Image: sputnikos/iStockphoto)

Manufacturers, logistics companies, pharmaceutical companies, and energy companies have something in common: They're using supercomputers to push the limits of research and discovery, as well as to answer questions that may not be possible or practical to answer using other means.

Organizations are using the cloud and PCs to solve the problems yesterday's supercomputers once solved. Cloud computing has also grown to encompass High Performance Computing or HPC in the cloud, and the providers of those products, services, and solutions are targeting research and scientific communities that have traditionally used supercomputers. As cloud solutions and supercomputers continue to advance, their use is not necessarily mutually exclusive. Some companies are working with universities and national labs to access the most powerful resources available, including companies that own their own supercomputers.

"We've got to get [companies] from where they are today to where we are in the massively parallel, multi tens of petaflop capabilities," said Jeff Nichols, associate laboratory director for Oakridge National Lab's (ORNL) computing and computational sciences, in an interview. "Companies in the automotive industry, the airline industry, the energy space, and science space want to work with us to solve their big science questions."

ORNL and the Joint Institute of Computational Sciences (JICS) -- which is a joint partnership between ORNL and the University of Tennessee -- each have a number of national class and leadership class computing resources with different architectures that allow them to solve different kinds and scales of problems. Titan is ORNL's big gun and the second most powerful supercomputer on the planet today. It's a Cray XK7 27 petaflop machine with 299,008 16-core AMD Opteron CPUs, 18,688 NVIDIA Tesla K20 GPU accelerators and 710 terabytes of total system memory.

In addition to having a wide range of hardware at its disposal, JICS optimizes packaged and proprietary software so it can run efficiently on supercomputers or in the cloud. In addition, JICS has a staff of 20 Ph.D.s in physics, chemistry, computer science, math and other fields who speak machine language. They help organizations understand what is possible to achieve computationally. And, because they're scientists, they also help companies advance the state of the art.

"As you begin to use computing, you start asking harder questions and need more computing. Novices hit a wall right away," said Tony Mezzacappa, JICS director and chair of theoretical and computational astrophysics and astronomy at the University of Tennessee, in an interview. "Knowledgeable folks [who] know what they're doing know how to scale across the full set of nodes on their machine, but eventually they hit a wall too. They want to solve a problem that requires more memory or more compute power to execute in a reasonable amount of time."

Here are six reasons why established companies and even startups are using supercomputing resources, and why your IT organization may want to consider it to meet your big data and business analytics needs.

Lisa Morgan is a freelance writer who covers big data and BI for InformationWeek. She has contributed articles, reports, and other types of content to various publications and sites ranging from SD Times to the Economist Intelligent Unit. Frequent areas of coverage include ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
1 of 7
Comment  | 
Print  | 
More Insights
Threaded  |  Newest First  |  Oldest First
User Rank: Ninja
6/29/2015 | 5:50:41 PM
Is that you, False-Positive?


One example is identifying fraudulent healthcare claims, which involves massive amounts of data. By bringing all the data into memory, it's possible to discover patterns that are not apparent if the claims are viewed in isolation[...]


LOL I smell job security for decades for plaintiff law firms bringing suit against skinflint ins. cos. denying legitimate claims because it spent millions on software and didn't take into consideration the cost of false-positive results and the ensuing litigation.

User Rank: Moderator
6/29/2015 | 7:10:40 PM
Re: Is that you, False-Positive?
Your point about false positives is well-taken.  Insurance companies often reject claims as a matter of course because it's in their best interest to do so.  A long, arduous process causes many claimants to settle for less than they might recover because they just can't weather the battle.  Knowing that, some claimants hire law firms hoping to to recover more than they would on their own, which works or doesn't depending on a number of factors.
CIOs Face Decisions on Remote Work for Post-Pandemic Future
Joao-Pierre S. Ruth, Senior Writer,  2/19/2021
11 Ways DevOps Is Evolving
Lisa Morgan, Freelance Writer,  2/18/2021
CRM Trends 2021: How the Pandemic Altered Customer Behavior Forever
Jessica Davis, Senior Editor, Enterprise Apps,  2/18/2021
White Papers
Register for InformationWeek Newsletters
The State of Cloud Computing - Fall 2020
The State of Cloud Computing - Fall 2020
Download this report to compare how cloud usage and spending patterns have changed in 2020, and how respondents think they'll evolve over the next two years.
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Flash Poll