5 Big Wishes For Big Data Deployments

We're all aware that the rise of big data is having a dramatic impact on the database market. Relational databases, which have been around since the 70s, were never designed to hold unstructured or semi-structured data, including social media posts, audio, video, sensor data and other digital flotsam that's growing dramatically.

Not surprisingly, the shortcomings of the relational database have led to the emergence of a variety of new and innovative data management technologies, such as NoSQL, a schema-less alternative that's popular with big data and real-time applications.

Add another contender to the list: DeepDB, a general-purpose database that performs simultaneous, real-time ACID transactions and analytics on the same data set. It's designed to manage both the structured data of relational databases, as well as the semi-structured and unstructured data associated with big data.

Many startups and established players are entering this field, of course, but Deep Information Sciences, the three-year-old company behind DeepDB, says its product is uniquely qualified to cover all the databases, if you will.

[ Does big data exacerbate mistakes? Read Big Data's Human Error Problem. ]

"We looked at the marketplace and saw that over the last 10 years or so, the database market has become pretty fragmented with a lot of special-purpose databases," said Mike Skubisz, VP of product management and strategy for Deep, in a phone interview with InformationWeek.

But these so-called specialty products, including NoSQL, NewSQL, column stores, streaming databases and so on, aren't one-size-fits-all solutions, he said.

"We set out to understand why a general-purpose database platform couldn't service, if not all, then a large percentage of these use cases that these specialty systems are being designed for," Skubisz added.

So Deep set out to build a general-purpose database engine with unique capabilities. DeepDB is a fully ACID-compliant transactional database that handles all relational database requirements, the company says.

"From that perspective, it can drop in and do what MySQL, Postgres, or Microsoft SQL Server can do with respect to relational," claimed Skubisz.

High performance is another of DeepDB's reported virtues. "Our target customer probably falls into a couple of different use cases where we add a lot of value. One is where high-performance transaction loads are important," said Skubisz, who added that DeepDB is "one to two orders of magnitude faster" than the competition, including the aforementioned MySQL, Microsoft SQL, and PostgreSQL.

Several Fortune 50 companies are beta-testing DeepDB, although Skubisz and Deep CEO and founder Kurt Dobbins, citing nondisclosure agreements, declined to name its enterprise customers.

In addition to fast transaction speeds, DeepDB promises high query-response rates. "Things that typically could take hours we do in minutes. Things that typically take minutes, we do in seconds," said Dobbins.

Company officials acknowledge that getting enterprises to commit a brand-new general-purpose database platform is a "tough sell."

"So we built an API layer on top of our platform that allows us to bring to market what we call database connectors," Dobbins said. "For example, we have a connector that allows us to connect our platform underneath MySQL. In that context, as far as any of the applications sitting on top of MySQL are concerned, nothing changes. It's just kind of faster."

DeepDB might indeed fill a void in the database market, but Deep isn't the only startup working on a general-purpose product that spans the relational and big data worlds. For instance, JustOneDB, a new relational database management system built upon PostgreSQL, handles billions of new transactions per day, supports analytics as well as fast transactions and queries, and runs on low-cost commodity hardware, the company says.