Facial Analysis: Wall Street's Next Big-Data Tool - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Big Data Analytics

Facial Analysis: Wall Street's Next Big-Data Tool

Researchers at the New Jersey Institute of Technology are analyzing facial expressions of corporate execs, as well as other data streams, to forecast stock market trends.

9 Google Apps Tips: Productivity Boosters
9 Google Apps Tips: Productivity Boosters
(Click image for larger view and slideshow.)

Facial expressions can provide a window into a person's unspoken thoughts. But can these nonverbal tells provide real-time insights for financial traders?

That's the goal of New Jersey Institute of Technology (NJIT) researchers, who are developing ways to filter and analyze corporate data to forecast future market trends. This big-data effort encompass a variety of information streams: including video of a CEO's presentation during a quarterly analyst call, in which analytics software scrutinizes the chief executive's facial or tonal changes; real-time speech-to-text conversion of the CEO's comments; and a live feed of the company's stock price and other market data.

This effort includes the work of Dr. James Cicon, an assistant professor of finance at NJIT's School of Management. Cicon focuses primary on empirical corporate finance.

[Google's revised version of CAPTCHA promises to make life easier for users. Read Google Gets Better At Spotting Humans.]

"Specifically, I look at boards of directors, CEOs, things like that," said Cicon in a phone interview with InformationWeek. "That's my big interest. I try to see if I can get more information to put in our financial models than perhaps [companies are] explicitly providing."

During a quarterly analyst call, for instance, real-time facial analysis would require a meticulous and speedy examination of a speaker's rapidly changing expressions, a task the NJIT system is designed to do.

"We track certain areas of the face," Cicon explained. "We look at the eyes -- the shape the eyes take during the course of the interview. We look at the eyebrows. We look for a downturned or widening mouth, or upturned lips."

NJIT's software extracts measurements of primitive emotional states, such as disgust, fear, and surprise. Its analytical capabilities are based in part on research conducted decades ago by renowned American psychologist Paul Ekman, who developed a comprehensive system for describing observable facial movements for emotions. "We can measure those facial expressions," said Cicon. "It's kind of straightforward if you have the machinery to look at a face and compare [it] with what Paul Ekman says a fearful face looks like."

Real-time measurement of emotions could prove useful in many fields, such as law enforcement and stock trading. A fearful-looking CEO on a conference call, for instance, might interest Wall Street traders hoping to profit from the manager's facial clues.

"You could take these primitive Ekman measures and combine them into higher-level measures as well," said Cicon. "For example, what we'd like to do is have a… lie detector running, or something like that. Does the manager believe what he's saying? This would be very useful to an investor."

Wall Street types could combine facial analysis with data from other sources as well. "You might do a live transcription where you take the manager's voice… and create a text stream from the words being said," Cicon said. "We can do content analysis of someone's speech to determine emotional states as well."

Given its ability to detect emotion states from a distance, facial analysis might have a future in law enforcement, too. "You can't hook someone up to a lie detector without their permission, so what we see is some type of a lie detector application. We wouldn't necessarily call it a lie detector. We might call a 'measure of sincerity.'

"Sincerity is the difference between what you're saying -- your words -- and what your face is expressing. A person who is saying one thing, but whose face appears to be telling us something else, might be considered 'insincere.' "

Facial analysis will soon find its way into real-world applications, Cicon predicted. "You have an academic paper written about something, and a couple of months later [stock] traders are using it as fast as they can. They recognize that academic research adds a lot of value to their work and gives them an advantage."

Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.

Jeff Bertolucci is a technology journalist in Los Angeles who writes mostly for Kiplinger's Personal Finance, The Saturday Evening Post, and InformationWeek. View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Joe Stanganelli
Joe Stanganelli,
User Rank: Author
12/8/2014 | 2:26:29 AM
Re: Speech Recognition already used when you call your insurance company
"If the system works correctly" being the key phrase.

Polygraphs can be faked out too (not to mention are notoriously unreliable).  There's no reason to think that a good actor couldn't fool a facial recognition system.
Joe Stanganelli
Joe Stanganelli,
User Rank: Author
12/8/2014 | 2:22:43 AM
Re: Speech Recognition already used when you call your insurance company
Or, for that matter, fake optimism.

It reminds me of competitive duplicate bridge.  ACBL rules dictate that all bids and plays must be made smoothly and unexpressively -- not only so that partner not be given unauthorized information, but also so that opponents (for whom, admittedly, such information would not be unauthorized) may not be "faked out."
Thomas Claburn
Thomas Claburn,
User Rank: Author
12/5/2014 | 5:07:40 PM
Re: Speech Recognition already used when you call your insurance company
I wonder if the system could be duped. You could then fake fear to convince facial analysts to sell when they shouldn't.
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

How CIO Roles Will Change: The Future of Work
Jessica Davis, Senior Editor, Enterprise Apps,  7/1/2021
A Strategy to Aid Underserved Communities and Fill Tech Jobs
Joao-Pierre S. Ruth, Senior Writer,  7/9/2021
10 Ways AI and ML Are Evolving
Lisa Morgan, Freelance Writer,  6/28/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Flash Poll