Updated, August 28, 2006

As business users, analysts and IT professionals pursue business intelligence nirvana, software tools get the most attention and excitement. After all, the BI tool is the face of BI, masking the data, technical infrastructure and information processes. Whether via a dashboard or other interface, the tool is the domain of IT's most important constituents: the users.

Yet for every successful BI project, you'll find it was achieved with different BI tools. So, if it's not the tool choice, what really makes--or breaks--BI? What do successful projects have in common? With business users' high hopes putting BI pros under intense pressure to create competitive advantages, you can't rely on technology alone to get you to the promised land. It takes more.

In this article, we'll discuss the seven "pillars," or factors that support the best BI implementations. To determine these factors, we talked with three award-winning organizations. In addition, Intelligent Enterprise conducted a survey with ASK (Analytic Solutions Know-How). Finally, we'll also look at what readers feel can help or hinder BI projects.

1. Measure Success

Companies struggle to measure BI success. Some measures, such as return on investment (ROI) show explicit, tangible benefits. Others point to "softer" advantages. These include whether users perceive the BI project as mission critical, how fully key stakeholders support the projects and the percentage of active users.

Many organizations go with ROI. However, ROI gives you a precise number based on imprecise inputs. If revenue increases by 10 percent, how much of that is directly attributable to the BI project? Market conditions, employee training and other factors such as a competitor going out of business could be just as responsible. Cost savings offer a more precise input--and have given companies much to brag about when it comes to BI. Here are some examples:

•Allstate Insurance, since it began a data and BI tool consolidation project in 2002, has saved tens of millions of dollars by eliminating redundant reporting systems.

•ENECO Energie, the third largest provider of electric and gas utilities in the Netherlands, saved $3 million in its customer contact center and billing department, raised conversion rates in campaigns and gained market share.

• 1-800 Contacts, a provider of direct mail contact lenses, estimates that its BI solution provides $50,000 per month in improved revenue, productivity and--most important--customer service.

Although it is generally difficult to calculate and apportion BI success using financial measures, organizations can tally the number of licensed and active users. Unfortunately, few companies do this--and BI vendors historically have not provided adequate tools to do so. However, high numbers of users tend to indicate success. In our reader poll, we found out that on average, only 24 percent of companies' total employees have BI licenses. But companies that described their projects as "very successful" report a much higher ratio (42 percent) of licensed users.

Differences in survey results highlight something interesting about expectations for how far BI can grow. Not surprisingly, nearly all the vendors will tell you that 100 percent of employees should have a BI license. However, The Data Warehouse Institute (TDWI), a leading industry association, reported in a recent survey that 41 percent of those whom respondents identified as "potential" BI users already have a license. This figure is nearly double what our ASK- Intelligent Enterprise survey says; we classified "all employees" as potential BI users.

Recent innovations discussed in Neil Raden's "BI Megatrends," BI vendors' partnerships with Google and other search technology vendors--will extend the reach of BI. Ten years ago, few would have said that every worker (let alone every teenager) needed a cell phone. Similarly, differences in perception among industry analysts, market prognosticators and vendors tell us that a range of opinions exist about BI's potential, and that many still do not view BI as part of the mainstream office tool portfolio.

2. Develop A Data Strategy

What's the single most important pillar in any BI architecture? It's obvious: the data. If you implement a BI tool on top of messy or irrelevant data, either the bad stuff proliferates throughout your organization or the BI tool quickly becomes shelfware. The overwhelming majority (78 percent) of respondents to our survey say that clean, relevant data is essential to BI deployment--a figure higher than any other factor.

Too often, data warehouse teams aim immediately for greater breadth when they should start with one subject area, establish high data quality and then move on. Allstate, winner of the 2005 Business Objects Customer Award, established a strong executive advisory group in the early stages of its BI project. The group identified and resolved all discrepancies between Allstate's old and new BI reporting environments. Business and BI managers together established guidelines for acceptable data quality.

Source systems can change frequently over a company's life. 1-800 Contacts found this to be a data-quality challenge. "We had to coerce the data into manageable sets," says Jim Hill, data warehouse manager. "However, we were focused on achieving data quality and tying it out to past reports. We had to explain the differences [in data] and show them to the business users, so they would trust the new dashboard." The company employs Microsoft SQL Server, Analysis Services, Reporting Services and tools from Dundas Software.

Even tougher than managing quality is determining what data is most relevant. All three of the companies highlighted here said they felt the data that users accessed contributed heavily to company performance--but for different reasons. At ENECO, BI efforts focused on learning which customers provided value and were better to retain. At 1-800 Contacts, order size and customer service metrics that focus on driving repeat business are key elements of their dashboard. As the screen on the facing page shows, the company created dashboard metrics that tie directly to call center representatives' performance bonuses. Early indicators allow representatives to take action and managers to provide additional coaching.

Thus, as you establish your BI environment, evaluate whether you are building a solution to replace old reporting systems that may be based on outdated business and data requirements. Use your BI strategy to create something core to business performance and provide the right data to support the effort.

3. Manage The Program Well

Nobody sets out to do a bad job. But unfortunately, BI projects can run amok, even with talented people at the helm. Project managers want so badly to be responsive to the business that they answer all requests, no matter how unreasonable or impulsive. So, click your heels three times and repeat the project manager's mantra: "There's time, resources and scope."

Stakeholders want everything yesterday. So, if you want to succeed, don't miss a deadline--not one. Budget upfront for required resources; this means you must be clear about what the business and IT sides must provide. And when requirements change, determine the impact on allotted time and resources. If requirements were ill-conceived from the beginning, scrap the deliverables and start over.

Too often, requirements are not prioritized adequately; instead, they are a compilation of everyone's wish lists. The scope keeps expanding to include more users, more subject areas and more capabilities. The BI team is setting itself up for failure--and setting up business users for disappointment. Establish data governance early so that data warehouse does not become synonymous with never-ending and expensive.

4. Ensure Executive Support

Everyone knows top-level support is vital. But how do you obtain it? One major BI analyst expressed his frustration that "without CXO edicts, political issues are showstoppers." In his experience, it's not easy to make sure that all levels of management buy into the enterprise BI vision, as individuals tend to lack the big picture. Rather than lead, managers react to what's happening in their own environment.

A BI project manager at a medical center related that "all the BI vendors come in and show the executives a bunch of eye candy. They make it sound easy when it's not. We end up with no funding or resources for our project." The moral is, don't let BI vendors' aggressive sales efforts set you up for failure. In our poll, INTELLIGENT ENTERPRISE readers told us that inadequate funding was a major obstacle to success.

"Business executives often have no idea what it costs for their analysts to manually gather information--and how vulnerable they may be because they don't know how their analysts get their numbers," the BI project manager says. "Based on what they see, an executive wonders why it takes a BI project six months to get going when his staff could give him 'answers' within an hour."

In some cases, executive support is frankly a matter of luck. You can improve your chances if you can bring in a tech-savvy executive who has had BI success elsewhere. Usually, however, support has to be earned--and re-earned. Even a small project success will gain trust and support. Ton van den Dungen, manager of ENECO Energie's BI Center of Excellence says that his company's executives initially frowned on BI. "Their attitude was 'we haven't seen one successful BI project; it's too expensive.'"

But, taking an entrepreneurial attitude, ENECO went ahead with a BI project in 2003 that involved manual extracts from source systems and Microsoft Excel pivot tables. Accounts receivable was the only subject area; the goals were to understand why receivables were high and identify opportunities to reduce them. The pilot, which cost only 350,000 euros, ended up saving ENECO 4 million euros (about $5 million). Success with the pilot project made it possible for the BI team to gain the support and funding it needed to develop a BI architecture that included a Microsoft SQL Server-based data warehouse and implementation of a suite of tools from Cognos. ENECO won the 2006 Cognos Customer Award.

Demonstrating success is crucial, but don't settle for less in garnering executive support. You should aim beyond BI, beyond IT and straight to the head of the boardroom. If the company's executive committee doesn't see IT as a strategic asset, BI will suffer from that perception as well. While our reader poll shows that CIOs are part of the business steering team in most companies (75 percent), other surveys report a far lower number. While some of these surveys may define such committees rather narrowly, we've seen numbers as low as 8 percent of CIOs serving on executive teams.

5. Choose Appropriate Tools

BI tools have gotten better, which means that selecting one is harder. Vendors now offer a broad spectrum of capabilities in one consolidated toolset. The differences are becoming less obvious. Thus, the key to success is to understand your users and recognize that different groups will have distinctly different needs.

As "Which Users Need Which Tools?" shows, at left, the number of users is often inversely related to what you're asking of the tool's capabilities. Many companies might, for example, have millions of users who only need access to fixed reports--but just a handful of report authors who demand advanced design capabilities, a scripting language and a rich query engine.

Ad hoc query tools have delivered the most success (see Listening Post, at right). These tools sometimes get a bad rap from industry pundits, who think that only power users need them. In reality, ad hoc query tools win out because of their flexibility and closeness to the business side. Today's ad hoc report often becomes tomorrow's fixed report, to be consumed by thousands (and fixed reports came in second in the poll at right). The difference is that a business user rather than an IT developer built the initial report; he or she tweaked it according to business requirements--and did it in a matter of hours, not days or months. "Our self-service approach has allowed us to teach our users how to fish rather than feeding them dinner," says Anthony Abbattista, Allstate's VP of Enterprise Technology Strategy and Planning.

6. Standardize On A Platform

Companies have multiple BI tools for a number of reasons. Before vendors started to offer suites, their customers were forced to buy different solutions from different vendors. For certain user segments and capabilities, this may still be necessary. Products from vendors that specialize in analytic applications (to handle campaign management or pricing, for example), visualization software and dashboards can be better than what you might get from a BI suite vendor.

For the most part, having multiple tools with similar qualities, typically bought from competing vendors by individual business units creates a big BI mess. However, under pressure to meet business needs, unsatisfied users will continue to seek out their own solutions. Yet according to Intelligent Enterprise's reader's poll, of the readers who consider their BI projects very successful, 73% classify their BI deployment as enterprise-wide and only 27% describe them as independent business unit implementations.

Standardization will put you on the road to BI success, but don't expect an easy drive. What do you do with old reporting systems? Once Allstate standardized on Business Objects products (with a data warehouse running on Oracle), the company offered incentives to encourage users to migrate to the new environment. As soon as major stakeholders were comfortable with the quality of the information presented in the Business Objects tool suite, Allstate pulled the plug on old reports. Other companies prefer a more conservative approach and let the old systems die a slow death. Our feeling is that if you are aggressive like Allstate, you'll focus resources on the new BI platform and eliminate multiple versions of data more quickly, saving money as Allstate did.

7. Align BI Strategy With Business

Too often, an "us versus them" mentality characterizes the relationship between business and IT sides. Rather than being the enabler, IT becomes simply the gatekeeper to data and technology. IT must change; it must understand what drives business performance. Is it revenue growth? Driving down costs? Providing better customer service? At a minimum, IT needs to be an active listener. Ideally, IT should be a valuable contributor that is involved in business strategy sessions.

ENECO's van den Dungen says, "We sit with the business so we know what's going on with the business. If you see the business as a partner, versus IT telling them what they need, there will be more agreement and the business will get much more involved [in IT decisions]."

BI is one of the hardest technologies to deploy because comparatively little of the challenge is about technology. The process demands a broader skill set and business focus than other aspects of IT. Note that of the seven success factors we've cited, only one relates exclusively to technology. As you embark on your BI project, get excited about the potential of the tools; they are the face of BI. But make sure you devote attention to the other factors, lest that BI beauty only be skin deep.

Debriefing

1-800 Contacts won the 2006 TDWI Best Practices Award. What's behind your success?

We aligned our first project with a call-center-incentive project. The BI team worked closely with call-center managers and used an agile software-development approach. There were some discrepancies between senior executives and call-center managers on how to show the numbers to agents. Executives didn't want to just give performance numbers--they wanted to show agents their numbers versus those of other agents to create a competitive environment to drive higher performance. The first pass was good: We got the data. With the second pass, the senior executives wanted a kind of "horse race" to get creative ideas going and better present information, so the front-end dashboard changed a lot. Now, an agent gets a quick visualization of how they are performing against themselves as well as what percentile they are in versus other agents. The system has helped change agents' behavior.

You worked closely with business users. How has that alignment influenced the BI project?

A close working relationship with the business users has been critical to our BI initiatives. The relationship between the business and IT was not always this good. Some projects were floundering, and initially, the difficulties slowed down the business side's interaction with IT. Business would shout, and IT would do a fire drill and throw something out there. Our new CIO established an agile software-development process with a much-improved governance model. We have much better communication, and business leaders are empowered to set and adjust priorities. The business side sees the data warehouse in a positive light, showing that IT can deliver something that gives business users immediate value and helps them make better decisions. We've also given analysts an OLAP data set that they can explore without IT involvement. This took pressure off IT to deliver individual reports.

What is your next BI initiative?

We are only beginning to leverage information in the marketing space with data mining and customer segmentation. We want to understand the effect of e-mail campaigns and are experimenting with different mining algorithms. Later in the year, we want to identify early or leading indicators and incorporate more monitoring and alerting.

Cindi Howson is the president of ASK and the author of BIScorecard ® product reviews. Write to her at [email protected].