What is automated decision-making?
Automated decision-making is the use of software to make, recommend or materially shape a decision about a person. The system may operate without human involvement, or a human may formally make the final decision while relying heavily on an automated score, ranking, flag or recommendation. Examples include credit approvals, fraud blocks, insurance pricing, recruitment screening, content moderation and eligibility decisions.
The importance of the term lies in the effect of the process, not only the presence of an algorithm. A system can significantly influence an outcome even when a person clicks the final approval button. The organisation should therefore examine whether the human reviewer has meaningful information, authority and time to challenge the output.
Why does automated decision-making create risk?
Automated processes can increase speed and consistency, but they may also reproduce biased historical patterns, rely on inaccurate data or make decisions that are difficult to explain. Large-scale automation can amplify a small design problem across many people before the issue is detected. Affected individuals may struggle to understand why an outcome occurred or how to correct incorrect information.
Risk is particularly significant when decisions affect employment, finance, healthcare, education, housing, insurance, access to public services or other important interests. Privacy, anti-discrimination, consumer-protection and sector-specific rules may impose additional requirements depending on the context.
How should automated decisions be governed?
Governance begins with a clear description of the purpose, data sources, logic, output and role of the human decision-maker. Teams should assess accuracy, bias, explainability, data quality, security, affected groups and the consequences of false positives and false negatives. The organisation should also document whether a less intrusive or less automated alternative could achieve the same objective.
Controls may include human review, decision thresholds, reason codes, appeal routes, monitoring, logging, quality checks and restrictions on use. Human oversight should be tested in practice; it is not meaningful if reviewers routinely accept recommendations without understanding them.
What information should be given to affected people?
Transparency should be proportionate to the significance of the decision. People may need to know that automation is used, the purpose of the process, the main factors considered, the role of human review and how to challenge or correct the outcome. Explanations should be understandable and should not rely solely on technical model descriptions.
When an individual disputes a decision, the organisation should have a route for reviewing the relevant data, logic and circumstances. The appeal process should be accessible and should allow a qualified person to change the outcome where appropriate.
Frequently asked questions
Is a recommendation an automated decision?
It can be. A recommendation may materially shape the final outcome if reviewers rely on it heavily or have little ability to depart from it.
Does adding a human always remove automated-decision risk?
No. Human involvement must be meaningful. The reviewer needs competence, authority, sufficient information and genuine freedom to challenge the system.
What is a legally significant effect?
The meaning depends on applicable law, but it commonly includes decisions that affect rights, contracts, access to services or similarly important personal circumstances.
Can automated decisions use inferred data?
Yes, but inferred information may be inaccurate or unexpected. Organisations should assess provenance, quality, transparency and the effect of using those inferences.
How should automated decisions be monitored?
Monitoring should track accuracy, subgroup performance, complaints, overrides, appeals, drift and unexpected outcomes, with defined thresholds for investigation or suspension.