The Fairness Compass: A Groundbreaking Step Forward for Trustworthy AI

Machine Learning has the potential to improve business performances in many areas. However, numerous cases of AI machine bias have been exposed in recent years, with new examples emerging on a regular basis.

The challenge of defining ‘fairness’

To tackle this, organizations usually focus on adjusting algorithms or data to satisfy some standard of fairness. But there is no universal definition of fairness: it involves multiple, conflicting notions. Sustainable solutions for fairer AI must go beyond technical methods and explain what the implemented fairness objective stands for and why this choice was considered most suitable for the given scenario.

The Fairness Compass: the northern star of AI

The Fairness Compass is an online tool developed by GETD (Group Emerging Tech and Data) which formalizes the selection process and makes identifying the most appropriate fairness definition for a given AI system a simple, straightforward procedure. It is based on a set of concrete questions regarding the nature of data, beliefs in its correctness, fairness policies, and whether the focus should be more on specificity or sensitivity of the model. These decision points guide AI practitioners through the complex landscape of existing fairness metrics towards the most suitable option. Moreover, recording the reasoning behind the decisions can serve as documentation to increase transparency.

Facilitating decision-making

Choosing the best fairness metric becomes a transparent, straightforward process. “From a business perspective, this facilitates decision-making by removing uncertainty. It enables deploying innovative technologies and allows substantial time saving,” says Boris Ruf, Research Scientist Lead Expert in Algorithmic Fairness.  

The Fairness Compass is also a significant step forward for machine learning. “I am very proud of the Fairness Compass: a new tool that guides us when choosing the right kind of fairness for a particular use-case and context. It is a significant step in our efforts to get our AI journey right, meaning adapting this exciting technology in a responsible manner,” says Marcin Detyniecki, Group Chief Data Scientist and Head of AI Research and Thought Leadership.

The Fairness Compass has been nominated for the Gartner Eye on Innovation award, several reports on AI regulation by international organizations also refer to the Fairness Compass, and it is now part of the World Economic Forum AI Fairness Global Library.

Click here to view the tool, or contact Boris Ruf: boris.ruf@axa.com.