Big Data's Surprising Uses: From Lady Gaga To CIA

Legal hold is a necessary annoyance for corporations, government bureaucracies and other large organizations. The term refers to the process of preserving all types of data -- including documents, email, voicemails, text messages and even social media posts -- that may be relevant in case of legal action.

But once an enterprise stockpiles what could amount to tens -- or even hundreds -- of millions of relevant pieces of information, how can it sift through the data quickly? After all, efficiency is essential during electronic discovery (e-discovery), the step in civil litigation where opposing parties exchange electronically stored information.

Eager to fill this market niche, several software vendors have developed document management systems designed specifically for e-discovery. These products include Symantec's Clearwell, Recommind's Axcelerate and kCura's Relativity.

"We all have a slightly different spin to it, but fundamentally we're doing the same thing," said Andrew Sieja, kCura founder and CEO, in a phone interview with InformationWeek.

[ Learn how college students propose using big data for legal discovery. See IBM Watson's Future: Students Have Winning Ideas. ]

According to Sieja, kCura's customers include top U.S. law firms, large corporations and government agencies such as the U.S. Department of Justice. "But our biggest segment comes from consulting firms that help corporations and law firms through large litigation projects," said Sieja.

KCura is currently managing about 27,000 active cases, and its software has roughly 73,000 users. In total, the company's software is managing close to 18 billion files, according to Sieja. "Probably a lot of the really messed-up stuff happening in the world right now, from a litigation standpoint, is being managed in our software," Sieja added.

KCura and its competitors use a machine-learning technology called predictive coding, which uses limited human input to enable a computer to "predict" how documents should be classified, according to attorney Matthew Nelson's Predictive Coding for Dummies book.

"Let's say you have a corpus of data with about a million documents," said Sieja. "Before you start your predictive coding project, you'll identify how accurate you want the review to be." Sieja offered this example: "I want to be 95% confident that we're going to be 100% accurate with a 3% margin of error." Based on these thresholds, the next step is to serve a statistical sample of the data to one or more expert attorneys, who review the documents and decide which are relevant to discovery requests.

"It's usually around the area of 1,500 docs," said Sieja. "Then we'll submit those decisions into the system, and based on their conceptual makeup we'll automatically classify all the other documents."

The process doesn't end there, however. A sample of the classified information is sent back to the experts, who review the material. If the computer-generated predictions don't match the accuracy target, the process is repeated again. "They'll continue this process until we're within the statistical range," said Sieja.

For corporations, governments and other large organizations, document management systems can expedite the e-discovery process, and save huge sums of money in document-review costs.

The rise of big data presents a challenge, however, and not just due to growing stockpiles of information. "In the world of e-discovery, a 100 million-document record set is very big for us. That's a lot of data for us to make sense of from an e-discovery standpoint," Sieja said.

Unstructured data poses another problem. "People are collecting data from phones now," said Sieja. And that's in addition to emails, text messages, Twitter and Facebook posts, and information from enterprise collaboration systems like SharePoint.

"If you look at where we're taking our product development roadmap, it's all about scalability and handling more data," Sieja noted.

He added: "A lot of the stuff that we do today is very reactive. More and more of these corporations are going to get more proactive in how they think about e-discovery."

Attend Interop Las Vegas May 6-10 and learn the emerging trends in information risk management and security. Use Priority Code MPIWK by March 22 to save an additional $200 off the early bird discount on All Access and Conference Passes. Join us in Las Vegas for access to 125+ workshops and conference classes, 300+ exhibiting companies, and the latest technology. Register today!