The way that we understand the world is through our bias and great AI is actually the best at using prejudice, identifying an object for example. However, certain bias can produce negative outcomes and the data on which it bases its determinations may be antithetical to what we want to see in the future. While we would want a machine-learning algorithm to learn what tumours looked like in the past, and to become biased toward selecting those kind of tumours in the future, the same is not true for learning what successful engineers and doctors looked like in the past and becoming biased toward selecting those kinds of people when sorting and ranking resumes.

This becomes harmful when we give the AI power such that it can impact the world. An algorithm’s prediction of high crime areas may result in more police being sent to those areas, with more crimes being detected as a result of increased police presence, ultimately confirming and compounding the prediction model in the case of machine learning.

The people who work with AI and ML believe there are a few areas which may help. One such area is algorithmic transparency where we can see, judge, and curate the data influencing the outcomes. Another is to transfer what we already have in law (no discrimination based on protected identities) to these models. After all, if black people are more statistically represented in crime reports it is not because they are black, but will be due to many other factors such as poverty, education, relationships, etc. These factors can all be taken into consideration without allowing skin pigment to have an influence.