Standard procedure in BI tool selections are bake offs. Customers provide a list of, oh, 100 to 200 requirements, and vendors must prove they can support those requirements. Only a lucky few make it to the final-round, proof-of-concept stage. It's a grueling process for customers and vendors alike, as fraught with tension and subtleties as the American Idol elimination process.

You might argue that BI requirements are much more objective than music performances, where taste in music is very subjective. In some cases this is true. For example, support for Linux is a clear yes/no. However, support for "multiple data sources" is open to interpretation, with all vendors responding "yes," but no two supporting it in precisely the same way. Ensuring a consistent understanding of requirements among all key stake holders is the key to ensuring a successful selection process. So as part of the course I teach at TDWI, we first start with those definitions. Only after this discussion do we launch into the bake off in the afternoon.Beyond consistent definitions, scoring vendors is particularly important for those requirements that are either subjective (ease of use and interface appeal , for example) or where definitions are subject to misinterpretation. Get those scores immediately following the demo to ensure that users don't forget what they saw, or worse, mix up vendors and products.

New for the next class (August 4th in San Diego if you can make it!) is a really nifty voting tool from Poll Everywhere. So just as Idol fans can text their favorite contestant, course attendees will be able to text which vendor best fulfilled a particular requirement. What's really cool is that the results are dynamically displayed as the votes come in. You can try a preview here, so just for fun, text in your favorite summer drink (lemonade, icedtea, beer, wine, or "marg" for margarita) to 99503.

I suspect it will be an enlightening addition to the class for attendees and vendors alike! It could also be a good complement to the traditional paper voting process we've used with customers. I'd welcome your experience on how you have captured scores and feedback during your own bake offs!

Regards, Cindi Howson, BI ScorecardStandard procedure in BI tool selections are bake offs. Customers provide a list of, oh, 100 to 200 requirements, and vendors must prove they can support them. Only a lucky few make it to the final-round, proof-of-concept stage. It's a grueling process for customers and vendors alike, as fraught with tension as the American Idol elimination process.