The best developers generally stay with a company when they believe strongly in its mission. Many of my colleagues deeply relate to what we do for a living -- in fact, several of my teammates see themselves as evangelists for our technology and services because they have also been victims of identity theft, online fraud, or shady mortgage brokers. These personal experiences motivate them to solve problems and work toward protecting consumers' personally identifiable information. For these colleagues, it's not about the so-called big data revolution; it's about simply making a difference.
A recent Edelman Berland survey identified the top technology policy issues on peoples' minds: In addition to keeping their data safe and private, people seek technology innovations that improve everyday life. Data -- made more intelligible by analytics and insights -- is the underlying foundation that makes peoples' lives more convenient, simpler, and generally better.
The next imperative for the evolving field of data science is the mission itself, a self-reflection to assert our better principles: Leveraging the information we analyze for the best purpose possible. Data is being used for global initiatives, in emerging enterprises, and in grassroots communities. Keeping an eye on such programs can help those who analyze data make a positive impact on society at large. Some of my most creative technologists get their ideas by reading The Wall Street Journal, or even a local newspaper, and asking themselves, "How can I help solve that problem?"
The drive to make a difference globally
I often meet with customers, and sometimes policymakers, who are examining new forms of data that can change industries on a global scale. In Amsterdam earlier this year, I met with some members of the European Commission to talk about telematics -- usage-based auto insurance, a concept that is taking off across Africa, Asia, and of course the US. We weren't sure how telematics would be received by politicians, but we imagined they might be reluctant to take a stand on the use of such data.
We were delighted to see these leaders immediately realize the business and consumer benefits. One commissioner said, "This should be great for consumers. They can see their driving behavior on a website and choose to improve their habits. Someone can actually choose to be a safer driver by observing empirical facts about themselves."
[Many factors contribute to the complexity of software development. There is no single solution. Read Software Development: Don't Ignore What Makes It Hard.]
In a world filled with dangerous individuals who fund terrorist activities and imperil lives, data has helped the world's banks learn more about their customers and share watch lists to flag signs of trouble. This is a tremendous initiative to help fight money-laundering all around the world. As a result of data analytics and technology, financial institutions can be more confident that they are doing business with people and businesses they know, and they can vet customers regardless of where they are.
Such collaborations across geographies are helping solve major problems in global business. Nations and NGOs are also joining in, tapping data to solve many of the world's other critical issues.
Data for good expands when businesses take action
Ten years ago, the buzzword "big data" was rarely heard, because only the largest companies had access to enormous volumes of data; very few companies aggregated or generated data in large amounts. These days, however, the proliferation of Internet-ready devices, smartphones, apps, tablets, and social media means that even small and medium-sized companies have data streaming at every moment about their products, services, and customers.
Businesses need to act swiftly, and taking action on reams of data requires a new kind of shorthand. Data scientists are using visualization to make underlying data sets more consumable and actionable, so what was formerly a 40-page report about an individual's track record in payments, for instance, can be reduced to not just a score but a trend. A dense report on a business's history can now be distilled into not just a rating but an interactive chart.
Findings based on data patterns have significant consequences on businesses. Good data can help reduce risks for both enterprises and consumers. It can help consumers who wouldn't otherwise qualify for a loan or a mortgage, for example. It can help ensure that healthcare funds go to retirees rather than fraudsters. It can help ensure that nobody else claims your tax refund.
Social graphs through Facebook and LinkedIn remind us daily just how few degrees separate us from the most obscure connections. Businesses with synergistic initiatives can come together faster
and vet each other more quickly, buoyed by information run through high-performance computing clusters that calculate risk in a given context.
Data for good begins on a grassroots level
When I reflect on my professional journey over the past 25 years, I owe much to my early years working in a tiny startup. I developed collaboration skills there, by wearing multiple hats so I could view a problem from different angles. Most productive technologists and data scientists can probably trace their success back to sharpening their ability to examine data critically and coming up with platforms, tools, and methods to aggregate and link the data quickly, meaningfully, and accurately. Such skills are more necessary than ever today as we encounter new challenges and data that require new tools and methods.
Harvesting actionable data insights for the greater good starts in one's own organization or community. Innovation and meetings of the minds between cross-functional areas are critical to nudge a theory into a practical and useful deliverable. Our company holds internal symposia, where we share what we've learned about the data sets and tools we are working on. This confluence of ideas across departments is innovation in the making. For instance, one person's hypothesis about certain individuals' access to government benefits in correlation to certain assets that the person owned was a catalyst that led to new tools for the prevention of fraud, waste, and abuse.
The discovery of such linkages between entities (such as research papers) can save time and eliminate lengthy processes. Recommendation engines powered by these kinds of techniques, for instance, can transform a search for the right information into a real-time suggestion. Recently, a scientist, desperate for one last piece of information to connect the dots, turned to Elsevier's Science Direct and instantly discovered exactly what he was looking for. His Twitter feed expressed his satisfaction: "I'd like to buy the maker of that recommendation engine a beer!"
Connections made at the personal or local level can fuel tremendous opportunities. For instance, US municipalities are aggregating crime rates, utility information, population rates, and other data to create "heat maps" that help homebuyers make purchasing decisions and governments address infrastructure and funding issues. The data has always been there, but depicting patterns in a visual way lets users act on it in unprecedented ways. Now individuals are helping fix communities, inspired by the ability to examine information in a way that reveals new possibilities for change.
Forging a path forward
Harnessing data for actionable insights requires talented data scientists who see non-obvious linkages in data. That could be one reason that no single approach to training creates a good data scientist; the next data scientist sitting beside you may be a former marine biologist. Data science is a mindset and a discipline, and new people are being drawn to the field. The use of data for good is happening at the local level, in businesses, and in the world at large. I encourage you to seize the opportunity to be a part of this movement.
If you just look at vendor financials, the enterprise storage business seems stuck in neutral. However, flat revenue numbers mask a scorching pace of technical innovation, ongoing double-digit capacity growth in enterprises, and dramatic changes in how and where businesses store data. Get the 2014 State of Storage report today. (Free registration required.)Vijay Raghavan is Executive Vice President and Chief Technology Officer for the risk solutions business of LexisNexis® and for Reed Business Information. In this position, he is responsible for technology development, research and development, information ... View Full Bio