5 Big Wishes For Big Data Deployments

Topology is a subset of mathematics that focuses on the study of shapes. The term dates back to the 19th century, but scholars were writing treatises on the subject at least a century before that. Topology isn't new, but using it to analyze and understand big data is a novel idea -- one that could empower business users to find value in very large data sets without having to consult data scientists or write algorithms or models.

That's the promise of Ayasdi, a Palo Alto-based startup that uses topological data analysis to quickly glean meaning from big data. "Our mission at our company is to build tools that everyone can use. We want to transform any person at a company into a data scientist," Ayasdi CEO and cofounder Gurjeet Singh told InformationWeek in a phone interview.

The company says its approach to analyzing big data is unique. Based on the work of Stanford University mathematics professor and Ayasdi cofounder Gunnar Carlsson, Ayasdi's enterprise-focused tools apply the abstruse concepts of topology to quickly identify relevant patterns in data.

[ What can we learn by tracking changes in human behavior in real time? Read Mashup Analytics: The Future Of Big Data. ]

In an introductory video to topological data analysis, Carlsson explains how three fundamental properties of topology combine in ways that enable the analysis of massive and varied data sets. The first of these properties is coordinate invariance, which means that topology measures properties of shapes that don't change, even as you rotate the shape or change the coordinate system in which you're viewing it.

The second property, according to Carlsson, is deformation invariance: The shape's properties don't change even if you alter its appearance (e.g., stretch or squash it). "Humans are really good at recognizing deformation invariant properties," Carlsson said in the video. "That ability is what allows us to recognize that a letter A is a letter A, no matter what font that letter is written in."

The third and final property is compressed representation: A sphere might represent infinite amounts of information, which is hard for humans to understand. But if you use an icosahedron to approximate the sphere's shape, the information becomes both finite and measurable. "It's still very much like a sphere, but now it's represented by a list, a simple list, consisting of 12 nodes, 30 edges, and 20 faces," Carlsson said.

The field of topological data analysis might not attract much interest outside of academia, but it appears to be drawing plenty of attention as the cornerstone of Ayasdi's big data tools. Fortune 500 companies in particular are taking note. Since January, Ayasdi has signed a number of large enterprises, including GE and Citi, as well as five of the top 20 global pharmaceutical companies, five government agencies, and two oil and gas companies, the company said.

In June, President Obama requested a private demo of the company's software after an Ayasdi intern developed an NBA player analysis that went viral.

Singh says Ayasdi's visualization tools are especially good at extracting insights from old data sets, a skill that makes them appealing to healthcare and pharmaceutical companies, which have been stockpiling digital information for many years.

Ayasdi's topology-oriented approach to data analysis is unique because few people understand the underlying technology, Singh said. "As far as we know, we are the only company capitalizing on topological data analysis," he added. "It's such a small community of people who even understand how this works. We know them all. A lot of them work at our company already."

Items from pills to power plants will soon generate billions of data points. How will this movement change your industry? Also in the new, all-digital Here Comes The Internet Of Things issue of InformationWeek: How IT can capitalize on the NSA's big data prowess. (Free registration required.)