Harvard Goes To School On Big Data - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Software // Information Management
Commentary
7/5/2011
02:21 PM
Doug Henschen
Doug Henschen
Commentary
Connect Directly
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Harvard Goes To School On Big Data

IBM Netezza appliance powers medical school's analysis of 10-million-plus patient records for drug safety research.

Web marketers probe clickstreams to find best prospects and most-profitable customers. Telcos examine call data records and social-network comments to identify influencers and potential defectors. And then there's Harvard Medical School, a big-data practitioner that is probing more than 20 year's worth of medical records to study the effectiveness and risks of various drugs.

Not to take anything away from the innovation and value of strictly commercial uses of big data, but Harvard's work holds lives in the balance. For example, Harvard's drug research spotted risks of teen suicide tied to certain antidepressants. Another study led to the withdrawl of a drug used in cardiac surgeries because it was shown to have higher risks than two safer alternatives. And yet another study done by the medical school led to new FDA warnings about a risk of violence among older patients using certain psychoactive drugs.

Harvard Medical research teams have been involved in pharmacoepidemilogy and pharmacoeconomic research for more than 13 years, and it has always been a data-intensive endeavor. With medical data now experiencing the same sort of explosive growth seen in other industries, Harvard recently stepped up to a state-of-the-art data warehousing appliance from IBM Netezza.

Most of Harvard's research centers on data from Medicare, Medicaid, and commercial insurance claims. Personally identifiable information is removed from this data before it gets into Harvard's hands, but there's still plenty of research-relevant detail on age, gender, race, previous conditions, indications, treatments, outcomes, and so on.

The medical school's databases have steadily grown over the years, and it has kept pace with a succession of technology deployments. As of last year, Harvard had multiple conventional relational database deployments running on big, proprietary servers running Unix. The total store covers more than 10 million patients and exceeded 15 terabytes of data, but with growing data and new techniques, the school knew it needed a faster and more capacious data warehousing and analytics platform.

"We were at breaking point, but it wasn't so much the quantity of the information as the richness of the data and the need to apply iterative algorithms," says Dr. Sebastian Schneeweiss, associate professor of medicine at Harvard Medical School and vice chief of the Brigham & Women’s Hospital.

The number of patient records needed for a particular drug test might be the same as required 10 years ago, but by richness, Schneeweiss means that for each patient record there are many new measures, such as body-mass index and LDL cholesterol level, that weren't tracked a decade ago. The iterative algorithms applied by Harvard's researchers are used to develop what are known as high-dimensional propensity scores, which cancel out biases in the sample by identifying relevant risk factors.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Previous
1 of 2
Next
Comment  | 
Print  | 
More Insights
News
Top 10 Data and Analytics Trends for 2021
Jessica Davis, Senior Editor, Enterprise Apps,  11/13/2020
Commentary
Where Cloud Spending Might Grow in 2021 and Post-Pandemic
Joao-Pierre S. Ruth, Senior Writer,  11/19/2020
Slideshows
The Ever-Expanding List of C-Level Technology Positions
Cynthia Harvey, Freelance Journalist, InformationWeek,  11/10/2020
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Why Chatbots Are So Popular Right Now
In this IT Trend Report, you will learn more about why chatbots are gaining traction within businesses, particularly while a pandemic is impacting the world.
Slideshows
Flash Poll