11 Ways IT Professionals Can Make Sense of IoT Data

Hoax callers are a special kind of jerk. At best they can cause their targets emotional distress. At worst they can cause or incite property destruction and even divert scarce resources from real emergencies where they are needed to save lives.

Technology can sometimes be applied to catch the criminal. Doing so casts technology specialists in the detective role by letting them trace calls to find their origins. Criminal pranksters have gained new tools to mask call origins, making these kinds of traces difficult or impossible in some cases.

The US Coast Guard is not immune to hoaxers. This branch of the US Armed Services monitors VHF channel 16, a marine radio frequency used for distress communications such as mayday calls that has also been a channel for hoaxers. By law, the US Coast Guard, which reports to the US Department of Homeland Security, is required to launch search-and-rescue missions in response to all mayday distress calls.

Gianfranco Palomba is an ensign in the US Coast Guard, working in its Research and Development Center, and the project manager for the organization's search-and-rescue hoax call project. He told InformationWeek in an interview that the US Coast Guard gets 130 to 150 prank calls each year.

Each of these calls results in a two-day search for the people in distress. A two-day search-and-rescue operation by the US Coast Guard costs taxpayers an average of $500,000 in terms of assets deployed for the mission -- personnel, helicopters, and boats.

"We've definitely seen an upward trend in these calls since 9/11," he told InformationWeek. The US Coast Guard's hoax call project has developed various technologies to track the origin of these calls to assist the investigation.

More recently, it has reached out to collaborate with academia on technology that can provide further insights into the origins and nature of such calls.

That's how recordings of these calls ended up on the desk of Rita Singh, a senior systems scientist at Carnegie Mellon University's School of Computer Science.

"The basic question was, 'Give us any information about this guy,'" Singh told InformationWeek in an interview. "The only thing we have is the recording of his voice."

Singh is a specialist in algorithms for computer speech recognition and speech processing and has studied this field for 19 years. Together with her team she applied a range of tools to the problem.

"It was like being thrown off the deep end of the pool. I applied everything I could to the analysis. My colleagues contributed, too."

Singh's team provided Homeland Security and the US Coast Guard with highly detailed information about the hoax caller and the environment this person was in when the call was placed. Because the case is still under investigation, the details Singh provided to investigators cannot be published.

[Is your enterprise already using artificial intelligence? Read Many Businesses Using AI Without Realizing It.]

Singh's research and application of technology to voice recordings can profile callers, revealing whether they are under threat, their health status, race, ethnicity, confidence, personality, truthfulness, and more.

"You cannot change how fast your muscles move, your vocal tract inertia, your lung volume, your skeletal structure," Singh said. "And all that influences voice. It's still ongoing research, but we are discovering things about how we can home in on those aspects of voice and speech that will allow us to make reliable and accurate estimations from the voice about the person's characteristics."

To accomplish this, Singh's algorithms look at the micro-properties (measurements made within fractions of a second) of both voice and speech.

"We are using machine learning algorithms to measure these micro-features, and we keep inventing new ones every day," she said. While others have pursued such research in the past, it's only today that the technology exists to extract some of these micro-features with great precision.

Singh provided InformationWeek with some additional examples of applications for this technology.

As for catching the hoax callers that have been costing the US Coast Guard and taxpayers so much money, all the cases are still ongoing, according to Marty Martinez, special agent with the Coast Guard Investigative Service, who spoke with InformationWeek in an interview. That means that the Coast Guard is not able to comment on any specific cases where this technology has been applied. But it's helping.

"Any time we can take new and emerging technology to help us do our job better, even if it's just a little bit, it's good for law enforcement," he said.

Palomba said this voice processing project is one of several collaborations by Homeland Security with academia designed to help investigations and protect the US.

His R&D group doesn't have the resources or project focus to center on this kind of research, but there's no question of its value.

"CMU's products and research have definitely enhanced investigations."

[Editor's note: This article was updated to mention that Gianfranco Palomba works in the US Coast Guard's Research and Development Center.]