What Is Bias in Machine Learning?

Key takeaways

In machine learning, bias occurs when AI models and algorithms include systematic errors that lead to unfair outcomes.

• Bias can develop due to biased or faulty training data, bias introduced by the humans who train the models, or a model’s own confirmation bias.

• If not identified and corrected, bias will find its way into AI systems, potentially leading to discrimination, ineffective decision-making, and poor customer experiences.

• You can address bias in machine learning through data improvement practices, algorithm testing and analysis, model design complexity, “fairness” in algorithm development, and human intervention.

Discover examples of bias in machine learning, where AI bias comes from, and what you can do to counteract machine learning bias. To deepen your understanding of machine learning, consider enrolling in the IBM Machine Learning Professional Certificate program. In as little as three months, you’ll have the opportunity to master up-to-date practical skills and knowledge that machine learning experts use in their daily roles. Upon completion, you’ll earn a shareable certificate to add to your LinkedIn profile.

What do you mean by bias?

Everyone, yourself included, has biases. Forming a bias helps you recognize patterns and decide how to move through life. Bias can help make quick decisions. For example, your prehistoric ancestors may have witnessed a community member get sick after eating a certain food and, as a result, developed a bias against it to protect themselves from possible poison. 

However, in the modern era, many of society's biases can be harmful and not helpful. This is particularly true when a person holds unconscious biases about marginalized groups of people, which leads to discrimination. Ideally, artificial intelligence would be free from these unconscious biases, relegating such errors as a relic of a biological brain. 

Unfortunately, AI models and algorithms can perpetuate human bias in many different ways, such as by taking as fact biases present in training data or in human error interacting with the model, for example. This reduces the accuracy of AI and can have real-world consequences for companies, individuals, and society. 

What is bias in machine learning?

Bias in machine learning happens when AI algorithms include systematic errors that lead to unfair outcomes, such as favoring one arbitrary group over others. This might include developing bias, for example, against marginalized individuals like people of color and women. 

This tends to occur because researchers train AI models with available data that often reflects the historical biases of the time or the biases of the researchers. In other cases, biases are continued through human error when interacting with AI analysis or data. Yet in other cases, biases exist because of the way that humans frame the problem they want to solve. For example, you might want to create an AI program that helps you locate the best restaurant in a city. First, you must decide what constitutes “the best,” which means the results may skew toward the foods you like the best instead of catering to a more diverse perspective.

AI bias can discriminate against individuals and give businesses and organizations inaccurate results. For example, imagine an AI model designed to pick the most creditworthy applicants for an apartment building. If the model demonstrated bias against non-white applicants, it would harm the applicants facing discrimination and the company, which cannot accurately determine which applicants would be the most creditworthy. Inaccurate artificial intelligence could lead to making poor decisions, offering a bad experience to your customers, or outright discriminating against groups of people.

Learn more: Bias vs. Variance in Machine Learning: What’s the Difference?

Example of AI bias

AI bias finds its way into systems if researchers and professionals do not take steps to watch for and correct bias. One example of bias in machine learning with real-life implications is a tool that Amazon released in 2014 and pulled in 2015. 

This tool was an AI application that could analyze the resumes submitted to a hiring manager and return the top five most qualified applicants. Amazon’s team trained the tool using data from 10 years of hiring history within the company. In analyzing this data, the AI tool trained itself that men were preferable candidates over women, partially due to the over-representation of men’s applications in the training data. When the program encountered a resume with words that indicated it was a woman’s application, such as being a member of a women’s organization, the AI penalized that application and ranked it lower than a resume without those terms. Amazon edited the program to avoid bias but ultimately ended the project. 

Machine learning bias could also lead to:

• Inaccurate medical diagnosis: When medical professionals use AI to diagnose disease or illness, the results could be inaccurate due to the underrepresentation of certain populations in the training or research data.

• Biases in search results: AI bias could lead search engine algorithms to decide what results to show based on demographics like your race or gender.

• Racial profiling in policing: When police and other safety officers use AI to determine what areas of their community require more police presence, they may rely on historical data that demonstrate patterns of racial profiling, thereby perpetuating that bias.  

How does bias in machine learning happen?

AI models can develop bias for several reasons, such as flawed training data, outdated historical data, or the model’s confirmation bias. Consider these types and sources of bias in machine learning:

• Training data bias: When bias exists in training data, it can appear in the AI model.

• Historical bias: Using historical data to train your AI could perpetuate bias that was common in that era.

• Algorithmic bias: Algorithmic bias refers to bias resulting from faulty training data. It can also occur if a programmer indoctrinates the algorithm’s decision-making ability with their own conscious or unconscious bias.

• Cognitive bias: Cognitive bias is an error based on the point of view of the human programming an AI, such as favoring US citizens over a global perspective.

• Confirmation bias: When an AI uses its own bias or faulty reasoning to confirm a bias as fact, it is called a confirmation bias.

• Exclusion bias: In some cases, it appears in machine learning after researchers exclude data they don’t believe is relevant, but would change the AI’s conclusion.

• Sample bias: When researchers don’t include enough training data for their AI, or if the training data they include doesn’t represent the population, it creates a sample bias.

Who works against bias in machine learning?

Different types of professionals work in machine learning and artificial intelligence, attempting to develop better algorithms. If you’d like to consider a career working to prevent bias in machine learning, three potential careers for you to explore include AI researcher, data scientist, and machine learning engineer. 

AI research scientist

Average annual salary in the US (Payscale): $127,939 [1]

Job outlook (projected growth from 2024 to 2034): 20 percent [2]

As an AI research scientist, you will use artificial intelligence to research and find solutions to real-world problems. You will determine the best strategies to create, develop, and train algorithms and apply AI analysis to complex problems. 

Data scientist

Average annual salary in the US (Payscale): $103,579 [3]

Job outlook (projected growth from 2024 to 2034): 34 percent [4]

As a data scientist, you will use data to recommend to businesses and organizations. In this role, you may work with or create AI algorithms to interact with data, gather raw data, prepare data for analysis, and present your findings. 

Machine learning engineer

Average annual salary in the US (Payscale): $125,201 [5]

Job outlook (projected growth from 2024 to 2034): 20 percent [2]

As a machine learning engineer, you will create, test, and troubleshoot machine learning applications for clients or to solve problems. In this role, you may also identify the proper data sets for training your algorithms.

How to address bias in machine learning

Just like discrimination in society, addressing bias in machine learning is a process that researchers and AI developers must continue to pursue. Important conversations are ongoing to determine the best path forward, such as defining what fairness will mean in artificial intelligence applications. 

Efforts to address bias in machine learning can be roughly sorted into measures you can take before running the algorithm, such as making sure your data is as impartial as it can be and measures you can take after processing the data, such as rebalancing the data against a standard of fairness. A third category also exists where you can add fairness constraints to the algorithm.

Other strategies for addressing bias in machine learning include:

• Improving your data

• Using data from diverse sources or representing a diverse population

• Testing and analyzing algorithms for biases proactively

• Designing more complex models

• Developing a thorough understanding of “fairness” in algorithm development

• Establishing AI policies that combat bias

• Keeping humans in the review process to look for errors

Stay current with trends in AI and machine learning

Join Career Chat on LinkedIn to stay current with the latest trends in your career field. Then, continue your learning journey with our other free digital resources:

• Hear from industry leaders: 6 Questions With a Google AI Research Director

• Watch on YouTube: Let’s Talk AI-What Are the Potential Risks and Benefits

• Bookmark online glossaries:Artificial Intelligence Glossary: Learn AI Vocabulary

Whether you want to develop a new skill, get comfortable with an in-demand technology, or advance your abilities, keep growing with a Coursera Plus subscription. You’ll get access to over 10,000 flexible courses. 

Build job-ready skills with Coursera Plus

Start 7-day free trial

• 
• 
• 
• 

Start 7-day free trial