How can AI be biased? And why does it matter?

If you have followed the discourse about Artificial Intelligence in the last few years, you have most likely come across news stories uncovering AI systems behaving in a “biased” way, describing gender or racial bias in AI-powered hiring processes or diagnoses.

According to the Cambridge Dictionary, bias is “the action of supporting or opposing a particular person or thing in an unfair way, because of allowing personal opinions to influence your judgment.”

At first glance, this appears to be a distinctly human trait and certainly not something an algorithm could display, given that it lacks human cognitive abilities (at least for now). Large Language Models (LLMs), which currently dominate the market, are advanced machine learning systems that generate text by predicting the statistical likelihood of words appearing together, not by forming opinions based on lived experience or socio-cultural immersion. This raises a crucial question: How does an AI system become biased?

In this article, we will explain what it really means for AI to be biased, the risks of biased AI systems, and what can be done to mitigate them.

‍

What Is Bias in the Context of AI?

When people speak of bias within an AI system, they usually refer to a phenomenon called algorithmic bias, however, bias also finds its way into AI through other avenues, also called non-algorithmic bias. In this section, we will break both of them down:

‍

Algorithmic Bias

Algorithmic bias refers to systematic errors in a computer system that produce unfair or prejudiced outcomes, arising largely from the quality of the training data, but possibly also from the design of the algorithm, or both.

‍

Machine learning models work by identifying patterns in training data and applying them to new, unseen data. For instance, a credit-rating model judges risk based only on past lending decisions, or a speech detection system flags offensiveness according to what has previously been labelled. Therefore, an AI system’s accuracy and fairness depend directly on the quality of its training data. Historical biases, discriminatory patterns, or incomplete and skewed datasets can lead to misrepresentation, underrepresentation, or omissions that the system then replicates causing biased and unfair outputs.

‍

These are the six most common cases of how bias can emerge from training data:

‍

• Historical Bias:

Historical bias arises when the training data is not representative of the current context in terms of time. Imagine a language model trained on a dataset from a specific historical period, adopting outdated language or perspectives.

‍

• Social Bias

Social bias occurs when machine learning models reinforce existing social stereotypes present in the training data, such as negative racial, gender or age-dependent biases. Generative AI applications can inadvertently perpetuate biased views if their training data includes data that reflects societal prejudices, historical or not. This can result in responses that reinforce harmful societal narratives.

‍

• Selection Bias:

Selection bias occurs when the data used for training a machine learning model is not representative of the population it is intended to generalise. This means that certain groups or types of data are either overrepresented or underrepresented, leading the model to learn patterns that may not accurately reflect the broader population or are selected without proper randomisation.

‍

• Reporting Bias:

Reporting bias arises when humans are involved in data collection or selection, often due to a tendency to under-report certain information. This results in incomplete datasets that fail to reflect reality and, in turn, produce biased AI outputs. For example, if you train an AI to evaluate restaurant quality using online reviews, the model will likely overrepresent extreme opinions, positive or negative, since people rarely write neutral reviews.

‍

• Group attribution Bias:

Group attribution bias occurs when individual traits are mistakenly generalised to an entire group during data labeling or selection. This bias can embed oversimplified assumptions into a dataset, causing AI models trained on it to produce distorted outputs. Consider the previous case of the COMPAS tool: group attribution bias may have contributed to disproportionately flagging Black defendants as high risk, by generalising the behavior of a few individuals to the entire group.

‍

• Feedback Loops:

Some machine learning models, such as batch or online learning models, continue to learn after deployment by adding observed results back into the training data. This can reinforce and amplify existing biases, creating runaway feedback loops that create a disproportionately extreme reality.  This could be detrimental to high-risk AI applications processing sensitive data and applied in sensitive contexts such as policing.

‍

Non-Algorithmic Forms of Bias

‍

• Implicit Bias:

AI systems are designed, implemented, and overseen by humans, who may carry implicit biases shaped by personal experience, cultural context, or societal norms. These biases can unintentionally influence every stage of the AI lifecycle from the selection and labeling of training data to algorithm design, auditing procedures, and decision-making outputs.

‍

• Automation Bias:

Automation bias occurs when people overly trust outputs generated by computer systems, even when those outputs conflict with other evidence or expert judgment. This is particularly concerning with AI, as large language models and other predictive systems can produce “hallucinations,” false correlations, or subtle biased outputs. Automation bias thus magnifies the risks of AI errors, making human oversight and critical evaluation essential.

‍

What are the risks of biased AI systems?

‍

AI-powered automation is increasingly used to scale operations and improve efficiency, accuracy, and objectivity. For example, AI systems now shape social media content, targeted advertising, hiring decisions, news distribution, and even government functions.  

‍

However, biases in AI training data can reinforce stereotypes or historical discrimination against women, vulnerable groups, and ethnic or religious minorities, or spread general misinformation. Because of this widespread integration, biased AI systems and outputs risk creating barriers to essential services and raise fundamental human rights concerns, including…

‍

• Unequal treatment in healthcare

For example, a 2025 study found that large language models (LLMs) used in long-term care in the UK and US tended to downplay women’s mental and physical health issues compared to men’s. This gender bias in AI-generated reports could result in misdiagnosis, inadequate treatment, and poorer health outcomes for women if not properly monitored and corrected.

‍

• Unequal access to social benefits

For example, Dutch tax algorithms incorrectly flagged thousands of parents for fraud in childcare benefit applications. Many affected parents had an immigration background. These errors, caused by biased algorithms and discriminatory data management, forced families to repay large sums and led to financial, social, and psychological hardships, including loss of homes, jobs, and child custody.

‍

• Infringement on the right to impartial policing and a fair trial

AI systems used in surveillance, policing, and criminal justice, such as facial recognition, biometric data, and predictive policing, can unfairly target specific groups.

‍

For example, the COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) algorithm is used in the United States to predict the likelihood of reoffending. Over the past decade, multiple studies have criticised COMPAS for producing racially biased outcomes. Although the algorithm is designed to ignore race, it disproportionately classifies Black defendants as high risk, resulting in more false positives for Black individuals and more false negatives for White individuals, a pattern known as proxy discrimination. Despite these findings, the tool continues to be used in the criminal justice system today

‍

• Unequal access to financial services

For example, Apple’s credit card algorithm was reported to give women lower credit limits than men with similar or higher incomes and credit scores, highlighting how AI-driven credit scoring can reflect historical biases, resulting in unequal financial opportunities.

‍

• Unequal access to the job market and fair salaries

AI systems embedded in hiring software, job-search platforms, and tools used by both recruiters and applicants for advice or evaluation are at risk of producing unfair and discriminatory outcomes. For instance, in recent years, several studies, including this one, have examined widely used LLMs for bias and found that when asked to provide advice on salary negotiations, they consistently recommended lower salaries for women than for men. These experiments also revealed unequal outputs linked to factors such as immigrant status and ethnicity.

‍

What tools do we have to address bias in AI systems?

‍

1. Development Tools and Practices

There are several tools and practices that can be leveraged to address bias across the AI lifecycle:

‍

• Diverse and Comprehensive Datasets:
  Careful data collection, selection, and preprocessing help prevent bias from incomplete or unrepresentative datasets. Developers can check data provenance, apply transformations, perform causal analyses, or use adversarial learning to promote fairness.

‍

• Synthetic Data: Synthetic data can help reduce bias in AI systems by filling gaps and balancing underrepresented groups in real-world datasets. It allows developers to simulate rare scenarios, remove biased correlations, and stress-test models across different demographics. By doing so, it improves fairness and generalisability while also enabling safe data sharing without exposing personal information. However, it must be carefully generated and validated, since biased synthetic data can still reinforce existing inequities.

‍

• Mathematical Debiasing: Formal techniques during and after the training of an AI model can help reduce bias in its outputs. Developers may reweight data, introduce fairness constraints in learning algorithms, or adjust predictions post-training to align with fairness criteria. However, this comes at the cost of the explanatory power of the model, since through these methods developers are, in essence, prescribing ideal outputs. In the worst case, this can create a circular reference, rendering the AI model effectively useless.

‍

• Regular Algorithmic Auditing: Internal or external audits assess an AI system’s fairness, accuracy, and safety by reviewing datasets, code, and outputs.

‍

• Human-in-the-Loop (HITL) / Human-on-the-Loop (HOTL):
  HITL integrates humans directly in decision-making, while HOTL involves human supervision. Both allow human oversight and intervention to prevent biased outputs and feedback loops.

‍

2. Laws and Regulation

Laws and regulation can establish binding standards that go beyond voluntary codes of conduct or ethical principles published by organisations such as the OECD or the UN. Regulatory frameworks like the European Union’s AI Act are specifically designed to mitigate risks, based on democratic principles and the protection of fundamental rights. By setting clear requirements, such regulation can ensure accountability, standardise processes for oversight, and establish liability when AI systems produce harmful or biased outcomes.

‍

3. Responsible AI Governance

Responsible AI governance based on explainability, transparency, and accountability is essential for addressing bias in AI systems:

‍

• Explainability provides insight into how an AI system reasons and produces outputs, making it possible to understand why a particular decision was made, identify biased outcomes, and determine where corrective action is needed.

‍

• Transparency involves openly sharing information about data sets, data collection methods, model design, and governance processes. This visibility helps stakeholders detect potential sources of bias, understand systemic risks, and foster trust and compliance with ethical and legal standards.

‍

• Accountability ensures that biased outcomes are addressed through mechanisms such as complaints procedures, audits, and independent reviews.

‍

Together, these principles demonstrate and enable a genuine commitment to socially responsible and reliable AI systems.

‍

4. AI Literacy

The use of AI tools among businesses and individuals has grown rapidly in recent years, but digital literacy, and AI literacy in particular, has not kept pace. A basic understanding of how AI works, along with its limits and risks, can help users recognise when outputs may be unfair or biased. Broader AI literacy can empower more stakeholders to challenge bias, engage in the discourse around responsible AI, and advocate for practices and policies that support equitable outcomes.

‍

5. Diversity in Tech and Governance

Increasing diversity within AI governance and development teams, for example regarding gender, social, ethnic, and economic backgrounds, can help create an environment that supports fairness and inclusivity. Involving people with varied perspectives throughout the AI lifecycle may also reveal blind spots and increase the likelihood that AI systems benefit people equitably.

‍

Conclusion

As AI becomes increasingly integrated into daily life, workplaces, businesses, media and government, its impact on essential services and fundamental human rights makes fairness a critical concern. Yet, AI systems are not neutral or free from bias because humans build and maintain them. Therefore, achieving a perfectly balanced and unbiased AI model may be impossible, given the diversity of human perspectives and values. Considering this limitation should also make us question where AI should rather not be employed. Discussions around bias in AI must therefore be ongoing and context sensitive.

‍

In a Nutshell

‍

What is Algorithmic Bias?

Algorithmic bias refers to systematic errors in a computer system that produce unfair or prejudiced outcomes, arising from the quality of the training data, the design of the algorithm, or both.

‍

How does AI become Biased?

AI can become biased through algorithmic and non-algorithmic factors. Algorithmic bias arises from flawed algorithms or biased training data, including historical, social, selection, reporting, and group attribution biases, which cause the system to produce unfair outcomes. Non-algorithmic bias occurs when human designers’ implicit biases or overreliance on AI outputs influence data selection, model design, and decision-making, further embedding prejudice into the system.

‍

What are the risks of biased AI systems?

Biased AI systems can reinforce stereotypes and discrimination, leading to unequal treatment in areas like healthcare, employment, and financial services. Such biases threaten access to essential services and fundamental human rights.

‍

How can we mitigate bias in AI systems?

Bias in AI can be mitigated through diverse, representative datasets, regular algorithmic auditing, and human oversight in decision-making. Strong governance, transparency, accountability, and adherence to laws and regulations help ensure fairness and correct biased outcomes. Promoting AI literacy and diversity in development and governance further supports equitable and responsible AI systems.

‍