AI Ethics & Bias
Why AI systems can be unfair, how it happens, and what is being done
Contents
Where bias comes from
AI bias is not usually a deliberate design decision — it emerges from the data and the feedback process.
Training data bias: If historical hiring decisions discriminated against women, and you train an AI on those decisions, it learns to discriminate. Amazon famously scrapped an AI recruiting tool that penalised CVs mentioning "women's" (as in "women's chess club"). The model had learned patterns from a decade of male-dominated hiring.
Representation bias: Models trained mostly on English text from Western internet sources will have weaker understanding of Indian languages, contexts, and values. "Normal" in the training data reflects the demographics of internet users circa 2020 — not humanity.
Feedback bias: When humans provide feedback to improve AI responses, their own biases influence what they rate as "good." RLHF (the key training technique) inherits the preferences of the rater pool.
Real-world examples of AI bias
Healthcare: AI models trained on primarily white Western patient data have been shown to misdiagnose or under-diagnose conditions in patients with darker skin tones. Pulse oximetry algorithms trained on lighter skin can give incorrect readings.
Credit scoring: AI credit models trained on historical loan repayment data reproduce past discrimination — areas that were redlined in the past continue to receive lower credit scores, not because of individual risk but because of systemic history.
Face recognition: Error rates for face recognition systems are significantly higher for darker skin tones and women. This creates unacceptable risk when used for law enforcement.
Language models: Most LLMs associate certain professions with specific genders by default (doctors as male, nurses as female). They also carry cultural assumptions that do not transfer well to non-Western contexts.
What is being done and what you can do
Diverse training data: Companies are actively working to include more representative data, including more languages, cultures, and demographics. Still a work in progress.
Bias auditing: Third-party organisations test AI systems for bias before and after deployment. This is becoming a regulatory requirement in the EU and increasingly expected elsewhere.
Explainability: Regulations like the EU AI Act require high-risk AI systems to explain their decisions. "The algorithm decided" is no longer acceptable for consequential decisions.
As a user: Test AI systems for the bias that matters to your context. If you are building a product for Indian users, test it with Indian names, Indian contexts, and Indian languages. Bias that does not affect English users may significantly affect yours.
AI tools are powerful — and like all powerful tools, they amplify both good judgment and bad. Human oversight of consequential AI decisions is not optional; it is responsible practice.