AI bias
Also known as: bias
What is AI bias?
Systematic skew in an AI system's output that produces unfair results, usually inherited from training data that carries human or historical bias. A hiring model trained on past hires can learn to favour the same profiles. Bias rarely announces itself. Finding it takes deliberate testing, which is why bias checks sit inside most AI governance.
Why it matters
The uncomfortable truth about bias is that you often cannot simply remove it, because “fair” has more than one definition and they pull against each other. Equalise a model’s error rate across two groups and you may worsen some other measure of fairness. No single setting satisfies every definition at once. That turns what looks like a technical fix into a decision about values, and it is not one engineers should be making on their own. Someone with authority has to choose which notion of fairness matters for this use, in the open, and be ready to defend it. Pretending the maths settles the question is how organisations end up enforcing a choice nobody admitted to making.
In practice
A lending model can be tuned so it approves the same share of applicants in each group, or so its mistakes fall evenly, but not both at once. Which one counts as fair is a policy call with real consequences for real people, so it belongs with someone accountable rather than buried in a model’s configuration. Writing down the choice and the reason for it is what makes the decision defensible later.
