Addressing AI Bias Head-On: It's a Human Job - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // AI/Machine Learning
07:00 AM
Mohan Mahadevan, VP of Research, Onfido
Mohan Mahadevan, VP of Research, Onfido

Addressing AI Bias Head-On: Itís a Human Job

Researchers working directly with machine learning models are tasked with the challenge of minimizing cases of unjust bias.

Artificial intelligence systems derive their power in learning to perform their tasks directly from data. As a result, AI systems are at the mercy of their training data and in most cases are strictly forbidden to learn anything beyond what is contained in their training data.

Image: momius -
Image: momius -

Data by itself has some principal problems: It is noisy, nearly never complete, and it is dynamic as it continually changes over time. This noise can manifest in many ways in the data -- it can arise from incorrect labels, incomplete labels or misleading correlations. As a result of these problems with data, most AI systems must be very carefully taught how to make decisions, act or respond in the real world. This ‘careful teaching’ involves three stages.

Stage 1:  In the first stage, the available data must be carefully modeled to understand its underlying data distribution despite its incompleteness. This data incompleteness can make this modeling task nearly impossible. The ingenuity of the scientist comes into play in making sense of this incomplete data and modeling the underlying data distribution. This data modeling step can include data pre-processing, data augmentation, data labeling and data partitioning among other steps. In this first stage of "care," the AI scientist is also involved in controlling the data into special partitions with an express intent to minimize bias in the training step for the AI system. This first stage of care requires solving an ill-defined problem and therefore can evade the rigorous solutions.

Stage 2: The second stage of "care" involves the careful training of the AI system to minimize biases. This includes detailed training strategies to ensure the training proceeds in an unbiased manner from the very beginning. In many cases, this step is left to standard mathematical libraries such as Tensorflow or PyTorch, which address the training from a purely mathematical standpoint without any understanding of the human problem being addressed. As a result of using industry standard libraries to train AI systems, many applications served by such AI systems miss the opportunity to use optimal training strategies to control bias. There are attempts being made to incorporate the right steps within these libraries to mitigate bias and provide tests to discover biases, but these fall short due to the lack of customization for a particular application. As a result, it is likely that such industry standard training processes further exacerbate the problem that the incompleteness and dynamic nature of data already creates. However, with enough ingenuity from the scientists, it is possible to devise careful training strategies to minimize bias in this training step.

Stage 3: Finally in the third stage of care, data is forever drifting in a live production system, and as such, AI systems have to be very carefully monitored by other systems or humans to capture  performance drifts and to enable the appropriate correction mechanisms to nullify these drifts. Therefore, researchers must carefully develop the right metrics, mathematical tricks and monitoring tools to carefully address this performance drift even though the initial AI systems may be minimally biased.

Two other challenges

In addition to the biases within an AI system that can arise at each of the three stages outlined above, there are two other challenges with AI systems that can cause unknown biases in the real world.

The first is related to a major limitation in current day AI systems -- they are almost universally incapable of higher-level reasoning; some exceptional successes exist in controlled environment with well-defined rules such as AlphaGo. This lack of higher-level reasoning greatly limits these AI systems from self-correcting in a natural or an interpretive manner. While one may argue that AI systems may develop their own method of learning and understanding that need not mirror the human approach, it raises concerns tied to obtaining performance guarantees in AI systems.

The second challenge is their inability to generalize to new circumstances. As soon as we step into the real world, circumstances constantly evolve, and current day AI systems continue to make decisions and act from their previous incomplete understanding. They are incapable of applying concepts from one domain to a neighbouring domain and this lack of generalizability has the potential to create unknown biases in their responses. This is where the ingenuity of scientists is again required to protect against these surprises in the responses of these AI systems. One protection mechanism used are confidence models around such AI systems. The role of these confidence models is to solve the ‘know when you don’t know’ problem. An AI system can be limited in its abilities but can still be deployed in the real world as long as it can recognize when it is unsure and ask for help from human agents or other systems. These confidence models when designed and deployed as part of the AI system can minimize the effect of unknown biases from wreaking uncontrolled havoc in the real world.

Finally, it is important to recognize that biases come in two flavors: known and unknown. Thus far, we have explored the known biases, but AI systems can also suffer from unknown biases. This is much harder to protect against, but AI systems designed to detect hidden correlations can have the ability to discover unknown biases. Thus, when supplementary AI systems are used to evaluate the responses of the primary AI system, they do possess the ability to detect unknown biases. However, this type of an approach is not yet widely researched and, in the future, may pave the way for self-correcting systems.

In conclusion, while the current generation of AI systems has proven to be extremely capable, they are also far from perfect especially when it comes to minimizing biases in the decisions, actions or responses. However, we can still take the right steps to protect against known biases.

Mohan Mahadevan is VP of Research at Onfido. Mohan was the former Head of Computer Vision and Machine Learning for Robotics at Amazon and previously also led research efforts at KLA-Tencor. He is an expert in computer vision, machine learning, AI, data and model interpretability. Mohan has over 15 patents in areas spanning optical architectures, algorithms, system design, automation, robotics and packaging technologies. At Onfido, he leads a team of specialist machine learning scientists and engineers, based out of London.


The InformationWeek community brings together IT practitioners and industry experts with IT advice, education, and opinions. We strive to highlight technology executives and subject matter experts and use their knowledge and experiences to help our audience of IT ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

Pandemic Responses Make Room for More Data Opportunities
Jessica Davis, Senior Editor, Enterprise Apps,  5/4/2021
10 Things Your Artificial Intelligence Initiative Needs to Succeed
Lisa Morgan, Freelance Writer,  4/20/2021
Transformation, Disruption, and Gender Diversity in Tech
Joao-Pierre S. Ruth, Senior Writer,  5/6/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Flash Poll