How Enterprises are Evolving Their NLP

Natural language processing, or NLP, is a subset of artificial intelligence (AI) that operates on text-, voice-, and video-based data. The NLP we typically encounter is in the form of an automated phone or chat attendant that attempts to answer all of our questions and then routes us to the right person, for instance, when we call a home improvement store. Or it is in the form of a "knowing" voice, such as Siri on an iPhone, that can tell us what the capital of Madagascar is, or where we can find the nearest Starbucks.

These examples show NLP working as voice-based automation, essentially serving as a "robot assistant" to get us where we need to go, and both of these examples are active IT NLP deployments.

Despite these established use cases, however, NLP has its challenges. For instance, can it ultimately succeed in being able to answer complex questions, or to understand a broader vocabulary of terms; or can it navigate the variety of linguistic accents and nuances that all contain meaningful content? Can NLP even punctuate correctly when you try to dictate a memo?

NLP can also fray users' patience, for instance when a customer repeatedly tries to give instructions to an automated attendant, and the attendant somehow can't grasp or execute what it is being asked to do.

Clearly, NLP is an emerging technology. It doesn't execute flawlessly in production. But for a number of companies, it appears to be "doing enough" to warrant implementation.

This begs the question, should NLP become a critical IT development platform? And where is it working? 

NLP as a development platform

There are a variety of NLP development platforms, and a significant number of them are open source. The advantage of NLP on open source is that it can be free for companies. Corporate IT also benefits from collaborative NLP improvements that are delivered by a worldwide software development community. The disadvantages are that support for these open-source platforms is not always readily available, and you can't always count on regular updates to distributions.

NLP platforms offer programming tools and libraries in Python and Java. This is good because many IT developers have experience in these programming languages. However, successful NLP projects require more than IT having the requisite development skills. IT and end users also need to know how to effectively integrate NLP with their business processes.

The telephone auto attendant is a good example.

How many of us have experienced frustration when we get caught up in an automated phone tree with multiple layers that seem to have no way out, and the NLP-based voice attendant doesn't seem to understand what we are talking about? Automated phone trees and routings have been available for years. They actually predate NLP. But these automated processes are often poorly designed. Even if NLP is added, it cannot fix a bad design. In these cases, an entire business process needs to be reinvented so that it has the customer experience in mind. NLP is then better positioned to make a positive contribution to that experience and process.

In other words, for an NLP deployment to work, IT not only needs the requisite technical skills. The company must have the business process skills so it can understand how to both optimize the business process and how to use NLP within the process to best advantage. If there is a failure in either of these areas, companies will not be able to derive the value they want from their NLP. 

Where NLP is working

Today, voice-based NLP works adequately on smart phones and other mobile devices, principally because users have quickly adapted to the limits of NLP, so they tend to tailor questions to the NLP so that the AI behind it can understand and process it.

Both voice and text-based NLP work marginally well in technical support applications on websites, but again they are limited. It doesn’t take them long to route users to frequently-asked-questions (FAQs) on the website that may or may not address the customer's specific questions.

Perhaps the area where NLP works best is in document dictation and analysis. In healthcare, for instance, there is an acute need for doctors to be able to dictate medical reports into electronic medical record (EMR) systems, so they don’t have to key them. The AI operating behind the NLP can review extensive data about patients and assist in diagnoses and treatments.

"NLP can recognize acronyms and biomedical entities," said David Talby, CTO at John Snow Labs, which provides NLP and AI solutions to healthcare. "The AI behind the NLP can extract important information about medications, and also uncover critical relationships between data points that could prove relevant to a patient’s status…A review of all available patient information can immediately detect that while a patient is short of breath, this condition only occurs when the patient is going up a flight of stairs. Observations like this offer physicians insights into patient conditions that assist them in better treating an illness or disorder." 

What we can expect going forward

According to Markets and Markets research, the global Natural Language Processing (NLP) market in healthcare and life sciences is expected to grow to $3.7 billion by 2025 from $1.5 billion in 2020.

Voice-based queries that trigger analytics probes of databases is an NLP sweet spot.

Other NLP-driven queries of AI engines are likely to follow, such as a semiconductor engineer voice-querying a database about which materials are best combined to make a certain type of conductor, or a logistics expeditor asking which route is best to get a shipment from Dayton to Sarasota.

In these cases, voice- and text-based NLP can be trained and adapted to a more finite set of users. Over time, we can expect NLP to grow in its ability to work with more linguistic nuances, and to expand to languages around the world.

Related Content:

Chief Data Officers Help Steer Digital Transformations

How to Take AIOps from a Promising Concept to a Practical Reality

What Some Customers and Employees Hate About Chatbots

What CIOs Need to Know About Graph Database Technology