Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.
Data Drift Happens: 7 Pesky Problems with People Data
Making sense of data associated with customers, citizens and other individuals may seem simple, but it’s possible that the records don’t tell the real story.
5 Min Read
Matthew Magne, SAS
People are like snowflakes. No two are alike. But, we all have similar characteristics – also known as personally identifiable information (PII) -- basic elements that organizations track about customers, citizens or any other representation of an individual.
But, people aren’t static entities. Lives change, and that leads to bad, or at least inconsistent, data. The problem manifests itself in marketing efforts that fail to reach the right customers. Or voter rolls filled with citizens who have moved, changed party affiliations or died. Or an insurance company that receives multiple claims about the same individual for the same incident.
Image: Pixabay/3dman
Luckily (or unluckily, depending on how you look at it), the problem isn’t new. Here are a few reasons why data on individuals has flummoxed organizations for decades.
Movement. According to the US Census Bureau, about one in nine Americans moved over a one year period. As a marketer, imagine more than 10 percent of your customer addresses changing in a single year. That’s happening, and while the percentage fluctuates, it’s a good reason that personal data remains dynamic.
Names. This may seem like a simple one, but in the case of marriages, it’s a change in personal data that happens frequently. What happens when a repeat customer signs up for the same service using a different name? You may create duplicate data if you can’t catch the similarities in other parts of the record. And one of the hardest things to do is to create a single view of the customer across all your systems. All of these factors can lead to fractured, inconsistent views of your customers.
Households. As people move, they often combine one family with another. In the case of a marriage or civil union, that can mean new financial arrangements or legal rights that an older database may not track. Or, if you get a new roommate, maybe there’s just another name on the bill. Any change in this status leads to data elements, like name or address, that need to be reconciled.
Channels. With the rise of social media, organizations have to track and relate even more channels of interaction. Thirty years ago, it was simply just a phone number and address – what fit into a now-quaint volume called a phone book. Then, organizations could track email, phone and store traffic. Add social media to the lists of channels, and it gets more complex. But, does email go away in favor of Snapchat? Absolutely not. It’s just another channel added to the mix.
Privacy. Customers can get online and order products, manage their accounts and receive timely service. Access is good. However, you have to figure out how to provide these services while avoiding a costly security breach.
Errors. People data can get out-of-hand because people are imperfect. People mistype data that they enter into self-service form. Or, a data entry clerk may misread a handwritten form or simply miss-key an item. Or they type an additional address or other personal information into the notes field where it can’t be sorted or found. All of this introduces the human element – and bad data – into your systems.
Processes. Personal data is the representation of an individual, but it is also the result of successfully executed business processes. If customer service staff isn’t trained correctly, they can input data incorrectly. If new customers are added to the sales force automation solution without a set of instructions designed to check if the customer already exists, you’ll have duplicates in your database. Now what happens if you change one of their addresses? Did you change both records? Which one is the best address? Bad data is often a symptom of a broken or failing process or a manual process that could be automated to ensure you have access to quality, trusted data
So, what’ the solution? First, simply understand that data drift happens. The trick is to deal with it proactively. That means you should keep better track of where your data resides. Create business rules and monitor it constantly to ensure it’s of high quality. Put in place automation where you can. And apply data quality to ensure there are less duplicates and correct addresses, helping you get that consistent view across different systems.
The negative ramifications of mishandling data increase every year with legislation like GDPR in Europe. These regulations could mean huge fines if data is not handled correctly. Beyond that, how can you deliver the best customer experience, if you don’t know your customer’s correct address, last name, or what products they bought both online and offline? It’s more important than ever to have a plan in place to handle your data, automate your broken business processes and apply data quality management to your business.
Every organization has a pool of data on customers, employees, citizens or other types of individuals. Making sense of that data may seem simple, but it’s possible that the records don’t tell the real story. With more data arriving each day – and more options for people to manage data outside of the enterprise IT structure – getting people data right is quickly becoming a must-have.
Matthew Magne is a TEDX speaker, musician, avid Catan player and data professional currently working as a thought leader for SAS in the Big Data Management space. A recovering entrepreneur and coder, Matthew has taught entrepreneurship in prison, launched multi-million dollar big data products, and spoken at numerous conferences on topics such as Big Data Analytics and Transforming Data Professionals into Data Rock Stars.
Matt is a member of the UNCW Marketing Advisory Board, the UNCW Data Science Advisory Board and a mentor at TekMountain, a co-working space and technology incubator. He received his BS, cum laude, in Computer Engineering at Boston University, has done graduate work in Object Oriented Development and completed his MBA at UNCW.
About the Author
You May Also Like
More Insights
