TDWI Selection Bias: It Depends Whom You Ask - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Software // Information Management
Commentary
2/20/2008
12:57 AM
Seth Grimes
Seth Grimes
Commentary
Connect Directly
Twitter
RSS
E-Mail
50%
50%

TDWI Selection Bias: It Depends Whom You Ask

The saying "There are three kinds of lies: lies, damned lies, and statistics" is attributed to Benjamin Disraeli, and it's nicely illustrated in a couple of Intelligent Enterprise reality-check photos from this week's TDWI conference. Check out the TDWI Image Gallery photos for clear illustrations of "selection bias": not TDWI's fault, but an effect to keep in mind when you assess formal and informal research findings.

The saying "There are three kinds of lies: lies, damned lies, and statistics" is attributed to Benjamin Disraeli, and it's nicely illustrated in a couple of Intelligent Enterprise reality-check photos from this week's TDWI conference.

Check out the TDWI Image Gallery photos posted by Intelligent Enterprise Editor-in-Chief Doug Henschen. Executive Summit attendees used the "dots method" to identify Important BI Technologies and Biggest BI Challenges. You get simple histograms showing what's hot and what's not. And you get clear illustrations of "selection bias": not TDWI's fault, but an effect to keep in mind when you assess formal and informal research findings.The Important BI Technologies results: 1 Dashboards/Scorecards; 2 Predictive Analytics; 3 Operational BI; and 4 BI Portals. My favorite topic, Text Analytics, is way down the list in a third tier. That's the reality check. (I hope this won't jeopardize the Text Analytics session planned for the August TDWI Executive Summit.) Open Source BI is even lower on the list.

These results aren't lies; no, they're statistics. Selection bias comes into play because a BI-themed session will attract folks grappling with technologies and solutions that are, well, in the solid BI mainstream. If you're out in front looking at text analytics, a BI-themed summit is probably not the place for you. Similarly, as I have reported, it's Java developers and not IT execs who are most interested in Open Source BI. It's a good guess that text-analytics types and OSBI types tend to self-select away from TDWI.

TDWI recognizes this effect. Philip Russom's 2007 report, BI Search and Text Analytics: New Additions to the BI Technology Stack, includes the explanation:

In an Internet survey conducted in late 2006, TDWI asked each respondent to estimate "the approximate percentages for structured, semi-structured, and unstructured data across your entire organization." Averaging the responses to the survey puts structured data in first place at 47%, trailed by unstructured (31%) and semi-structured data (22%). Even if we fold semi-structured data into the unstructured data category, the sum (53%) falls far short of the 80-85% mark claimed by other research organizations. The discrepancy is probably due to the fact that TDWI surveyed data management professionals who deal mostly with structured data and rarely with unstructured data. All survey populations have a bias, as this one does from daily exposure to structured data.

You'll find similar, similarly understandable examples of selection bias in the Biggest BI Challenges dots results, where #1 is "Gaining consensus on data definitions." Yup, that sounds like a BI/DW manager or exec speaking; the rest of us are stuck in our cubes, waiting for the execs to get back from TDWI in Vegas, keen to consensusize us.The saying "There are three kinds of lies: lies, damned lies, and statistics" is attributed to Benjamin Disraeli, and it's nicely illustrated in a couple of Intelligent Enterprise reality-check photos from this week's TDWI conference. Check out the TDWI Image Gallery photos for clear illustrations of "selection bias": not TDWI's fault, but an effect to keep in mind when you assess formal and informal research findings.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

Slideshows
IT Leadership: 10 Ways to Unleash Enterprise Innovation
Lisa Morgan, Freelance Writer,  6/8/2021
Commentary
Preparing for the Upcoming Quantum Computing Revolution
John Edwards, Technology Journalist & Author,  6/3/2021
News
How SolarWinds Changed Cybersecurity Leadership's Priorities
Jessica Davis, Senior Editor, Enterprise Apps,  5/26/2021
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Slideshows
Flash Poll