6 IT Job Hunting Dos & Don'ts

Hadoop vendor Hortonworks raised $100 million in its initial public offering on Friday. And in its first day of trading, the company's stock rose 65% from $16 to $26.38 per share at the close of trading.

It's a nice story for investors, but why should CIOs be swayed by one company's fortunes on Wall Street? For starters, Hortonworks and other Hadoop distributors have succeeded in raising money only because there's growing evidence that CIOs, particularly those at big companies, are sold on Hadoop.

It hasn't hurt that tech analysts are confirming that Hadoop adoption is for real. Forrester Research, for example, wrote last month that Hadoop is now "a must have for large organizations." Yes, there are still plenty of "dazed and confused CIOs" when it comes to forging big-data strategies, Forrester grants, but it sees the future in proven, mission-critical deployments of Hadoop at Wal-Mart, Fidelity Investments, Sears, Verizon, USAA, Cardinal Health, Wells Fargo, Procter & Gamble, Cablevision, Nasdaq, AutoTrader, Netflix, and Yelp.

[Want more on big data? Read Top 5 Big Data Trends Of 2014.]

During its IPO roadshow, Hortonworks detailed two real-world Hadoop use cases that likely opened some eyes, both because they're compelling and because they involve very mainstream companies. Here's a hint of what these two companies have achieved.

Case No. 1: Home-improvement retailer
The goal for this multi-channel retailer was to get a 360-degree view of customer behavior across previously siloed data sources including point-of-sale, in-store design centers, contractor systems, geo-fencing systems (used to track traffic patterns), and the retailer's online store. High-scale and highly variable data types in this mix (read, not suited to relational databases) include mobile data, clickstream data, log files, and geo-fencing data.

Spotting correlations across all these sources with the aid of Hadoop, the retailer has developed a clearer understanding of customer buying habits. Geo-fencing data, for example, can show time spent in various departments correlated with what was purchased at checkout. The retail can tell when customers are just there for an isolated purchase, when they're contractors buying in bulk, or when customers are purchasing items tied to a kitchen-remodeling project (having recently used design-center services). The retailer has also developed a better understanding of purchasing across multiple store locations, according to Hortonworks.

The payoff in these projects has been the ability to develop more personalized promotions, including product bundling and up-sell and cross-sell recommendations. These have produced gains in both revenue and margin, according to Hortonworks, and the program is in its infancy. The retailer is currently working on improving operational efficiency, with better visibility into buying patterns expected to yield better supply chain planning, stocking, and merchandizing.

Case No. 2: Auto insurance provider
A well-known auto insurer has a safe-driver program that relies on devices that plug into onboard car computers to capture data on driving habits. More than 1 million customers have joined this program, and the insurer has thus far captured data on more than 10 billion miles of driving.

Hadoop economics made it the natural choice for storing all this sensor data, and the insurer is correlating it with the conventional driver demographic, driving-record, and risk data it has relied on for decades. The combination yields more accurate risk calculations down to the individual customer. Customers get better rates, and the insurer has benefited more broadly (and financially) because it has developed a better understanding of risk. Accuracy will only improve as the data store grows.

Hortonworks didn't have permission to name these companies, but it's pretty easy to guess that it was talking about Home Depot and Progressive Insurance, both of which have been publically identified as Hortonworks customers.

Half empty or half full?
According to the InformationWeek Analytics, BI, and Information Management Survey 2015, which we'll be releasing later this week, more than half (58%) of 374 respondents involved with information management technology selections say they have no current or planned use of Hadoop.

As shown above, only 4% say they're using Hadoop extensively, up from 3% last year, while 18% say they use Hadoop on a limited basis, up from 12% last year. Another 20% now plan to use Hadoop, up from 19% last year.

There are those who will see in this data a glass less than half full, with nearly 60% not even experimenting with Hadoop. But a closer look reveals that large companies -- those with 10,000 or more employees -- are more sold on Hadoop. Among respondents in this group, 9% say they use Hadoop extensively, 32% use it on a limited basis, and another 29% say they plan to use Hadoop. Less than a third (30%) say they have no plans to use Hadoop.

Even before last week's IPO, Hortonworks and its competitors have had no trouble convincing even industry insiders that the innovators are embracing a big-data future in which Hadoop will play a key role. HP, for example, invested $50 million in Hortonworks in July. Google Capital invested $80 million in MapR in June. And then there's the $740 million Intel invested in Cloudera in March.

What these industry insiders know is that with mobile apps, clickstreams, log files, sensor data, and other high-volume and highly variable data types on the upswing, CIOs must take steps toward managing and understanding this data. CIOs at larger companies are clearly waking up to the possibilities.

Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.