Data Governance is Broken - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management
Commentary
12/14/2020
07:00 AM
Barr Moses, CEO & Co-Founder, Monte Carlo; Keyur Desai, Data & Analytics Executive
Barr Moses, CEO & Co-Founder, Monte Carlo; Keyur Desai, Data & Analytics Executive
Commentary
50%
50%

Data Governance is Broken

So how did we get here? Here are the three main ways data governance is failing us, along with some advice on how to address those areas.

Image: Sikov - stock.adobe.com
Image: Sikov - stock.adobe.com

Over the past several years, data governance has emerged as more than just a trendy buzzword. With the passage of GDPR, CCPA, and other industry, government, and healthcare compliance measures, data governance has become a corporate necessity. Yet many chief data officers cite data governance as a major hurdle for their organization.

So how did we get here? Here are the three main reasons why data governance is failing us:

1. A manual approach is no longer practical.

While we’ve made great advancements in areas such as self-service analytics, cloud computing, and data visualization, we’re not there yet when it comes to governance. Many companies continue to enforce data governance through manual, outdated, and ad hoc tooling. Data teams spend days manually vetting reports, setting up custom rules, and comparing numbers side by side. As the amount of data sources increase and tech stacks become more complex, this approach is neither scalable nor efficient.

While data catalogs often market themselves as the answer to data governance, many data leaders find their catalogs lacking in even the most rudimentary aspects when it comes to manual requirements (for example, lineage tracing and data quality checks).

In some organizations, these operations take up a significant amount of time in manually mapping upstream and downstream dependencies -- not to mention the maintenance work required to keep this up to date. Instead, companies should turn to ML and automation to reduce manual work done by data teams. Our advice: let ML do all the heavy lifting and let your team to focus on the things only they can do.

2. Data is ubiquitous; data governance is not.

Data is everywhere and everyone wants to use it. Teams across the enterprise are voraciously collecting and consuming data to make smarter business decisions. As a result, companies are hiring data engineers and data analysts by the bushel, creating additional data assets and pipelines. Analytics that could only be pulled on a weekly basis are now accessible by the hour.

For many companies, increasing the speed of data innovation is critical to survival. While data infrastructure and business intelligence tools have advanced to support this innovation over the past several years, DataOps has lagged behind with most DataOps solutions like data quality alerts and lineage tracking being manual, one-dimensional, and unscalable.

One of the ways in which DataOps and solutions can catch up is by drawing on concepts from software engineering. Many of the problems we face in data are in fact problems that have been solved in engineering, security, and other industries.

3. Data privacy and security are front and center. For everyone. 

What do Uber, Marriott, Facebook, and Equifax have in common? All four companies have recently been the subject of massive data breaches, costing them millions in SEC settlement fees and eroding valuable customer trust.

While very public companies have faced the most scrutiny as a result of data breaches, not even tornado sirens are immune; increasing regulations and media attention to hacks and breaches are making data privacy and security a priority for everyone. Companies big and small need to take these concerns seriously through a robust data governance strategy.

Instead of keeping these conversations among a small and siloed team of experts, CTOs, CDOs, and VPs should put together a cross-functional data committee that sets security and privacy KPIs and keeps the entire organization accountable to meeting them.

Make way for data governance 2.0

Automation/ML, next-gen DataOps, and data privacy and security are not only foundational to innovation, but critical to the future of data governance. Data governance 2.0 will emerge from the convergence of these three trends, taking center stage for not just CDOs but entire organizations.

If you’re struggling with data governance, know that you are not alone. While there’s plenty of room for improvement, we’re excited to see which new approaches rise to meet this challenge. As data leaders, it’s on us to drive home the importance of this topic, both in our companies and the community at large.

After all, if data governance is a top-level concern, it’s about time we treat it like one.

Barr Moses is CEO & Co-Founder of Monte Carlo, a data observability company. Previously, she was VP of customer operations at Gainsight. Prior to that, she was a management consultant at Bain & Company and a research assistant at the Statistics Department at Stanford. She also served in the Israeli Air Force. Barr graduated from Stanford with a B.Sc. in Mathematical and Computational Science.

Keyur Desai is a global enterprise data management, data monetization and analytics executive. He was most recently Chief Data Officer for TD Ameritrade, where he led a globally dispersed team with full budget responsibility.

 

The InformationWeek community brings together IT practitioners and industry experts with IT advice, education, and opinions. We strive to highlight technology executives and subject matter experts and use their knowledge and experiences to help our audience of IT ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
News
Can Cloud Revolutionize Business and Software Architecture?
Joao-Pierre S. Ruth, Senior Writer,  1/15/2021
Slideshows
10 IT Trends to Watch for in 2021
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/22/2020
News
How CDOs Can Build Insight-Driven Organizations
Jessica Davis, Senior Editor, Enterprise Apps,  1/15/2021
White Papers
Register for InformationWeek Newsletters
The State of Cloud Computing - Fall 2020
The State of Cloud Computing - Fall 2020
Download this report to compare how cloud usage and spending patterns have changed in 2020, and how respondents think they'll evolve over the next two years.
Video
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Slideshows
Flash Poll