9 Reasons Why Personalized Marketing Still Isn't Accurate

As we enter 2016, big data initiatives are becoming more the mainstream than cutting-edge, and many CIOs are now heading up efforts that go well beyond what they dreamed of implementing a decade ago.

Many organizations are still in transition as they look at the data they have, the data they want to add, the products and services they want to create with that data, and what infrastructure and tools they need to accomplish those goals, according to Randy Bean, CEO and managing director at NewVantage Partners, a management consulting firm that works with C-level executives at many financial services firms in the Fortune 1000.

"The past five years have been about big data," he told InformationWeek. "Organizations want to bring in new sources of data. They want to create new information-based products and services."

This week, Bean's firm released the results of its 4th annual Big Data survey of C-level executives at a smattering of Fortune 1000 companies. Of the 44 companies participating in the survey, many of them are financial services firms and healthcare-related companies, which are two vertical market specialties for the consulting firm.

While the survey, titled "An Update on the Adoption of Big Data in the Fortune 1000" focuses on a small sample, and is described by the NewVantage as anecdotal, it does gather insights from some very high-level executives, including CEOs, CIOs, and chief data officers at big, influential companies. Participants included American Express, MasterCard, Capital One, Bank of America, Morgan Stanley, and Charles Schwab.

Bean shared the results of the survey with InformationWeek. He spoke with us about the questions and concerns that he is hearing from executives at these firms today as they seek to implement big data initiatives.

A Quick History Lesson

Most data efforts in the early 2000s began as "CRM-driven" initiatives as organizations sought to leverage the customer data collected, Bean said. That effort gave way to compliance-driven and regulation-driven efforts in the age of Sarbanes-Oxley and other regulations that put pressure on organizations to achieve better data quality.

That environment has shifted again, starting about five years ago, to encompass big data, Bean said.

Today's organizations are working in two types of environments. On one hand, they have an investment in their legacy enterprise data that resides in CRM systems and their data warehouses that are well-suited for compliance requirements. Then they have added new streams -- or raging rivers -- of data, capturing every detail that can be captured about customers and the transactions they make.

Leveraging the 'Big' in Big Data

For consumer financial services companies, capturing the data behind every transaction of every consumer provides a deeply granular view about where individual consumers shop and what they may buy next. That sets up a whole new realm of possibilities.

"You could be in a position to introduce new kinds of offers," Bean said. "We are in an early stage of figuring out how to do that in a way that is most meaningful to the consumer -- matching services to customers based on needs."

[Looking for more perspectives on the current state of big data? Read Big Data Moves From Hype To Reality.]

Organizations today are often pursuing those goals by implementing big data environments that coexist with the data warehouse, according to Bean. Organizations are currently looking at what information is better suited to what environment.

"There are certain things that a data warehouse is suited for, like data compliance" or operational reporting. "Big data is more about discovery environments and looking for patterns … Right now there is a value to both environments."

Another factor that comes into play between the data warehouse environment and the big data initiatives is cost.

"One of the original premises of big data was it was much more cost-effective than traditional data environments," Bean said. "And that will likely be the case." But the ramp-up in big data has included an upfront investment that has made it financially comparable to what companies already have in place. As companies realize value from their big data investments, the cost will drop. But we haven't quite gotten there, according to Bean.

Big Data Survey Results

While Bean frames the survey as anecdotal due to its limited field of participants, it still provides some interesting insights into how big data efforts have shifted among some big firms within the Fortune 1000.

The survey revealed that 62.5% of respondents said they now have at least one instance of big data in production, nearly double the number of those (31.4%) who reported the same in 2013. Some 26.8% of firms surveyed reported that they will invest more than $50 million in big data by 2017. That's up from just 5.4% of them who said they invested more than $50 million in 2014. Also, 69.6% of those surveyed said they now view big data as very important or critical to their business success.

Chief Data Officer

This year's report shows that 54% of companies surveyed have created the role of chief data officer compared to just 12% in 2012. A full 20% of respondents say that the CDO is the executive with primary ownership for big data initiatives and 14.3 say that the CDO is the primary executive sponsor for big data.

Business Factors Are the Key to Big Data Project Success

What's the biggest factor in business adoption of big data? Not surprisingly, 33.9% (the largest percentage) of those surveyed said that it's business and technology cooperation. Strong business sponsorship was cited as the most critical factor by 23.2% of the respondents. Technology leadership was cited by 5.4%, and no one surveyed cited technology selection as a critical factor.

Along similar lines, 37% of those who participated in the survey (the largest percentage) said that the ability to develop greater insights into business and customers was the biggest driver of big data investment. The next biggest driver cited by respondents included advantages gained from speed -- faster time-to-answer, time-to-decision, time-to-market -- as indicated by 27% of the respondents.

Big Data Hits Mainstream

"In 1995 through 1999, people were rushing to take advantage of the Internet. Now people just use the Internet," Bean said. "They don't say 'Hey I'm using the World Wide Web now,' they just do it. We are entering that phase where big data will just become accepted and a normal part of things. It will take a few more years but that's where we are headed."

**Elite 100 2016: DEADLINE EXTENDED TO JAN. 15, 2016** There's still time to be a part of the prestigious InformationWeek Elite 100! Submit your company's application by Jan. 15, 2016. You'll find instructions and a submission form here: InformationWeek's Elite 100 2016.