How Data Collaboration Solves Big Data's Biggest Problems - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management
Commentary
11/25/2019
07:00 AM
Tal Golan, Chief Strategy Officer
Tal Golan, Chief Strategy Officer
Commentary
50%
50%

How Data Collaboration Solves Big Data’s Biggest Problems

Imagine the possibilities of combining data across markets and industries without the cost and expertise barriers of implementing traditional data sharing.

There are a growing number of data collaboration solution providers competing to solve the IT management challenges of enterprise data fragmentation. This is the problem where siloed teams all want to work on large data sets, including data from other teams, and so copies of the data are made and shared, resulting in increased costs of storing and managing multiple copies of the same data in an enterprise.

Enter data collaboration, a relatively new approach that is shaking up big data management and is at the cutting edge of data-driven decision making. Data collaboration allows multiple parties to share access to their data, which gives all parties a more holistic view of the project and enables deeper insights. Not only do teams reap the benefits of having immediate access to a wider array of data, IT departments benefit from reduced data fragmentation and lower management costs while simplifying the deployment architecture.

Image: maxsim - stockadobe.com
Image: maxsim - stockadobe.com

These platforms also offer significant value-added features, such as rapid integration and development, as well as advanced data visualization. At the same time, they eliminate the need to distribute and control additional copies of data. Data collaboration does away with the necessity of developing APIs in applications or planning complex solutions to make the data fully accessible -- all of which can take months before teams can even get their projects started. Instead, users can directly leverage the platform tools to build solutions rapidly and turn around a digital service without the need for skilled software developers to write extensive code. Think of it as what CMS products did for web publishing, or what cell phone cameras have done for content creation. It produces instant results by removing all the heavy lifting.

While the benefits are apparent for enterprises, what about organizations or researchers that want to collaborate with third parties or partners outside of their controlled networks? With data collaboration, the biggest boon to organizations is the added security features for controlling access to the data. Since access is provided through the platform itself, the granularity of access can go down to the data cell level. This allows owners to decide which data are shared to maximize collaboration, while ensuring conformity to regulatory and security requirements. There is no need to worry about what a third party will do with personal data: the owner is always in control and no copies are ever distributed.

In business, data collaboration is the key for achieving the next level of strategic decision making. Imagine you want to expand your department store offerings to increase revenue. You already have your own data on what customers are buying in your store, but what are they buying that you don’t already sell? With data collaboration, you could partner with a payment system provider and find out what customers spend their money on. Critically, not only would you know what your customers are buying from your store versus elsewhere, but you could also see what purchases they are making on the same day. That additional data from a collaborator could tell you if you should open a grocery section and whether it can effectively compete by saving your customers an extra trip. The advantages that data collaboration offer to businesses is obvious.

Magical unicorn?

By now, data collaboration may start to sound like a magical unicorn. How is this practical or even possible? How can you achieve insight when you don’t possess the data? The reality is that with large data sets, you don’t gain insight by looking at the actual data. There’s just too much of it. Instead, you formulate queries around the questions that need to be asked. It’s the query results that will provide the insightful answers. This is already a common practice with data that you do own, so imagine the possibilities of combining data across markets and industries without the cost and expertise barriers of implementing traditional data sharing, integration and security controls.

All this added insight can be achieved without having to know who the individuals are and what each person is purchasing. As a collaborator, you can ensure that personally identifiable information is not exposed, and your customers’ privacy is maintained, because it is the queries that produce insights. These queries can be shared with other collaborators for mutual benefit. As an IT leader, you can reduce your complexity and costs for information systems and data storage. As a program manager, you can accelerate outcomes by rapidly prototyping and productizing solutions. It’s a revolution for big data with the potential to transform how we make use and make sense of data without compromising on security and privacy.

Tal Golan is a technology investor, entrepreneur and former chief strategy officer at Verb Technology.

The InformationWeek community brings together IT practitioners and industry experts with IT advice, education, and opinions. We strive to highlight technology executives and subject matter experts and use their knowledge and experiences to help our audience of IT ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Slideshows
Reflections on Tech in 2019
James M. Connolly, Editorial Director, InformationWeek and Network Computing,  12/9/2019
Slideshows
What Digital Transformation Is (And Isn't)
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/4/2019
Commentary
Watch Out for New Barriers to Faster Software Development
Lisa Morgan, Freelance Writer,  12/3/2019
White Papers
Register for InformationWeek Newsletters
State of the Cloud
State of the Cloud
Cloud has drastically changed how IT organizations consume and deploy services in the digital age. This research report will delve into public, private and hybrid cloud adoption trends, with a special focus on infrastructure as a service and its role in the enterprise. Find out the challenges organizations are experiencing, and the technologies and strategies they are using to manage and mitigate those challenges today.
Video
Current Issue
The Cloud Gets Ready for the 20's
This IT Trend Report explores how cloud computing is being shaped for the next phase in its maturation. It will help enterprise IT decision makers and business leaders understand some of the key trends reflected emerging cloud concepts and technologies, and in enterprise cloud usage patterns. Get it today!
Slideshows
Flash Poll