5 Big Wishes For Big Data Deployments

Teradata announced Wednesday that it's forging deeper, in-database processing ties with Revolution Analytics and a new partnership with Fuzzy Logix. What's missing from the picture is SAS, Teradata's longstanding partner and co-pioneer of in-database processing techniques that have shaved hours and, in some cases, days off of advanced predictive and statistical analyses.

The highlight of Wednesday's news is the deeper partnership with Revolution whereby that vendor's library of analytics based on the open source R programming language has been adapted by Teradata to run with scalable parallel processing power within the Teradata database. Teradata and other partners have previously supported Revolution's software, but Teradata says it's the first to bring efficient, parallel processing to the vendor's High-Performance R offering.

"We used to run the software at the node level, where you were reliant on the data scientist to determine parallelism," Teradata product and services marketing manager Chris Twogood told InformationWeek in a phone interview. "Now we're driving results in parallel, whether you're doing data manipulation, descriptive statistics or predictive algorithms."

[ Want the scoop on the latest big analytics move? Read SAP Buying KXEN For Predictive Analytics. ]

The bottom line of automated, parallelized processing is faster performance and greater accuracy. Teradata isn't promising specific levels of improvement, but Twogood said Teradata has seen cases where Revolution jobs that previously took three hours took three minutes with the new approach and others where 12-hour analyses were cut down to five minutes. The approach ensures higher accuracy because analyses run against all data without relying on interim aggregations.

Deeper support for Revolution's portfolio will make a broad base of R-based algorithms available to Teradata customers. SAS, too, has made selected R-based algorithms available to run on its software, but they don't run in parallel in partner databases and the choice of R algorithms is limited.

Teradata is not backing away from or diminishing its current partnership with SAS, Twogood insisted, it's simply responding to customer demand. "Our customers are demanding R and they're demanding more parallelism," he said. Despite SAS's R offerings, he added that customers "want native R, not algorithms exposed through SAS, because they're looking for alternatives for some of their analytics."

In an unrelated interview earlier this week, SAS marketing executive Malene Haxholdt acknowledged the interest in R.

"Open-source analytics is bringing a lot of value to customers, and we are constantly adapting to that and making it easier to integrate R into the SAS world," she said. "With that said, the use of R is for fairly high-end analytics users because it typically involves coding. We're focused on making our environment for advanced analytics easier to use."

Haxholdt said SAS's next move toward easier analysis will be the release of Visual Statistics, a drag-and-drop and point-and-click-oriented option for advanced analysis work.

[ Editor's Note: SAS contacted us after this article's publication with the following comment: "This is not a snub in any way," said Scott VanValkenburgh, SAS Senior Director of Alliances. "SAS and Teradata both partner with competitors and we expect that to continue. SAS priorities include in-memory technology and Hadoop integration from an in-database perspective. We continue to focus on in-database functionality. SAS and Teradata see in-database and in-memory solutions as business critical for customers, as evidenced by Teradata's recent announcement of an appliance specific to SAS analytics. Demonstrating that our relationship is as strong as ever, the market can expect a new SAS-Teradata announcement in the next quarter." ]

Teradata's in-database partnership with SAS dates back to 2007, when the two companies partnered on a scoring accelerator. The partnership deepened in 2011 when Teradata became one of two vendors (along with EMC) to offer in-database processing for SAS High Performance Analytics software. There was no real option to extend the relationship at this time, but SAS can't be happy to see not one but two new options for Teradata customers to work more closely with competitors.

The new relationship with Fuzzy Logix will see more than 600 of that vendor's algorithms running in parallel on Teradata. Together with what's available from Revolution and Teradata's own library of 300-plus algorithms, Teredata claimed it now offers "the largest, most complete in-database analytic library" with more than 1,000 choices of algorithms.

The default choice before the in-database approach was developed was moving data sets over to dedicated analytic servers for analysis. But with ever-growing data stores, data movement gets time consuming and the dedicated servers tend to be too underpowered for efficient, high-scale data analysis.

"In-database analytics is no longer an emerging trend, but now an absolute requirement to meet the processing speed and data volume demands," said Scott Gnau, president of Teradata Labs, in a statement. "Teradata empowers its customers to push beyond the traditional limits of analytics by bringing the analytics to the data."

Teradata also announced on Wednesday the latest release of its database, Teradata 14.10. The update delivers previously promised upgrades including Intelligent Memory in-memory processing, improved access to data within Hadoop clusters and enhanced workload management capabilities.