Data Quality's Threat to Democracy

Data Quality expert Larry English (catch his keynote address at the next TDWI) has claimed data quality is the second biggest threat to human kind, after global warming. When I first read this statement, I thought it was hyperbole, meant to engage readers. But English makes some compelling arguments. Yesterday's cover story in USA Today is yet one more case to support Larry's dire claim: data quality problems threaten this year's presidential election process.Eligible voters are being rejected because data doesn't match other government databases such as social security and motor vehicle administration records. Guess the Feds aren't any better at master data management than corporate America! In Colorado, for example, a full 20 percent of voters have been rejected because of these data quality problems. I don't know about you, but I don't have too much faith in manual, back-up ballots for such voters.

Data quality is one of the biggest challenges to successful BI and one for which there is no easy fix. As highlighted in Chapter 7 of Successful Business Intelligence, some visionary companies such as Dow Chemical have gone so far as to appoint a "quality czar" who has authority not only over the BI environment, but also over the source systems where data quality problems usually start. Not all organizations have that vision or that luxury, however. FlightStats - like a voter registrar - is at the mercy of other organizations who create the data. For FlightStats, achieving a high level of data quality has been a multi-year process of understanding each sources' nuances, measuring data quality, and continuous improvement.

It would have been ideal if USA Today's article included practical tips on how not to get rejected from the voter database, or how we as voters exacerbate the data quality problems. I'm notoriously bad at sometimes using Cynthia, Cindi, and if a clerk fills in the form, no doubt, it's Cindy. If only I thought about the big picture repercussions!

What strategy has most helped your company improve its data quality?

Regards, CindiData Quality expert Larry English has claimed data quality is the second biggest threat to human kind, after global warming... Yesterday's cover story in USA Today is yet one more case to support Larry's dire claim: data quality problems threaten this year's presidential election process.