What is algorithmic bias?
Algorithmic bias is a systematic pattern in the outputs or operation of an automated system that disadvantages people or groups, produces materially different error rates or reinforces unfair historical conditions. Bias can appear even when developers do not intend discrimination and even when a model performs well on average.
The term covers several different problems. A dataset may underrepresent a population, historical decisions may contain discrimination, labels may reflect subjective judgement or a seemingly neutral variable may act as a proxy for a protected characteristic. Bias can also arise after deployment when a model is used in a context different from the one in which it was tested.
Where does algorithmic bias come from?
Bias can enter at every stage of the AI lifecycle. Problem definition may prioritise the wrong outcome. Data collection may exclude relevant groups. Labelling may be inconsistent. Model optimisation may favour overall accuracy while hiding poor performance for a subgroup. The user interface may encourage over-reliance on a score, or operational teams may apply the output without an effective appeal route.
Historical data deserves particular attention. A model trained to imitate past hiring, lending, policing or healthcare decisions may reproduce inequalities contained in those records. Removing protected characteristics does not necessarily solve the problem because other variables can remain strongly correlated with them.
How is algorithmic bias assessed?
Assessment begins by identifying affected groups, potential harms and the decisions influenced by the system. Teams should evaluate data quality and representativeness, compare performance and error rates across relevant subgroups, test edge cases and consider whether the selected fairness measure fits the context. No single metric proves that a system is fair.
Quantitative analysis should be combined with qualitative review. Subject-matter experts and affected stakeholders may identify risks that are invisible in a dataset. Testing should also consider intersectional groups, because a system may perform adequately for broad categories while failing for people who belong to multiple groups.
How can algorithmic bias be reduced?
Mitigation may involve improving data collection, correcting labels, changing model objectives, removing or controlling proxy variables, applying constraints, adjusting decision thresholds or redesigning the business process. Strong human oversight, meaningful explanations, appeal procedures and post-deployment monitoring can reduce harm but should not be used to justify a fundamentally unsuitable system.
All mitigation choices involve trade-offs and should be documented. Organisations should record the fairness goals selected, the evidence reviewed, known limitations and the residual risk accepted by the responsible owner.
Frequently asked questions
Does removing protected characteristics eliminate bias?
No. Other variables may act as proxies, and historical patterns can remain embedded in the data. Fairness requires evaluation of outcomes and context, not only removal of specific fields.
Can an accurate model still be biased?
Yes. Overall accuracy can hide substantially worse outcomes for a subgroup. Performance should be examined across relevant populations and types of error.
Is bias always illegal?
Not every statistical difference is unlawful, but discriminatory effects may trigger legal obligations depending on the jurisdiction and context. Legal review should accompany technical testing in significant use cases.
Can human review solve algorithmic bias?
Human review can help only when reviewers have meaningful information, authority and time. Humans may also reproduce or amplify bias, especially when they over-trust automated recommendations.
How often should bias testing be repeated?
Testing should occur before deployment, after material changes and during operation. Changes in users, data or social conditions can produce new patterns even when the model itself has not changed.