An essay by Robin Pocornie

Artificial intelligence (AI) is intended to make our lives easier. However, it often disadvantages certain groups – such as those from diverse racial and ethnic backgrounds, as well as individuals based on their sexual identity or age. This occurs when the data used in machine learning reflects existing societal biases. Robin Pocornie experienced this firsthand as a student. Since then, the now-scientist and speaker has been advocating for fairer AI-based systems. Here, she explains why more diverse datasets alone won't solve the problem. 

Facing unfair outcomes caused by racial bias

In 2020, at the height of the covid-19 pandemic, I became an AI justice advocate. Not by choice, nor due to my academic pursuits in computer science at the time, but because a mandatory facial detection algorithm failed to recognize my face. The online, at-home examination software implemented by my university was unable to detect students with darker skin tones. As a result, I faced delays and obstacles entering my exams, unlike my white peers. This compelled me to report my unfortunate and, one could say, hurtful experience to the university, especially as reports claimed more students worldwide were encountering the same issues.

After running through several hoops (and months) to find the correct university office to report the issue to, the university decided against honoring my request to cease the use of the online exam software. I have since made it my mission to fight for fair and just algorithms, and alter the way organizations, governments, and the private sector look at implementing these systems. In 2022 the Netherlands Institute for Human Rights ruled in an interim judgement that I had provided enough evidence to suggest algorithmic discrimination had occurred. However, after the university had provided counterevidence, the Institute concluded in October 2023 that although the specific software might be discriminatory, it had not been conclusively proven that discrimination took place in my particular experience.

Datasets trained to establish our AI models have been shown to favor the reality of wealthy, European and North-American perspectives. This perpetuates race and gender stereotypes, which negatively affects systems combatting these inequalities. For example, text-to-image generators prompted to produce images of people in various occupations, yielded images of white males for high-income occupations and images of darker-skinned females in the low-income occupation category.

Additionally, AI can perpetuate gender bias in recruitment. This was exemplified by Amazon’s AI driven hiring model, which favored male candidates for technical roles due to historical data reflecting gender imbalances. This bias further highlights the need for transparency and fairness in AI systems to prevent discrimination and ensure equal opportunities for all candidates.

The prevalent solution proposed to address AI bias is to diversify the datasets used to train these systems. I believe that while these calls for more diverse datasets are well-intentioned, they are insufficient. We must adopt a different perspective that fundamentally rethinks AI development. This reimagination of AI development requires prioritizing justice and equity over mere technological capability.

3 Ways to fairer algorithmic decisions

I am often invited by companies and organizations to share my expertise on this ethical AI perspective. The solutions I propose to the growing bias in our decision-making models can be achieved at the developer’s desk and in the boardroom, as they take both the technological and organizational aspects of dealing with AI into account. What I find most important is to put visibility on the impact of AI, which often remains hidden from public view and can potentially create unforeseen ramifications.