AI Bias & Fairness
Why It Matters for Security and Risk
Bias in AI isn't just an ethics problem — it's a compliance risk, a legal liability, and a reputational threat. For regulated industries, biased AI outputs can trigger enforcement actions, lawsuits, and regulatory scrutiny.
Types of AI Bias
Data Bias
The training data doesn't accurately represent the population the model will serve.
| Bias Type | Description | Example |
|---|---|---|
| Selection bias | Training data drawn from a non-representative sample | Hiring model trained only on data from one demographic |
| Historical bias | Training data reflects past societal inequities | Credit model learns to deny loans based on zip code (proxy for race) |
| Measurement bias | Inconsistent data collection across groups | Medical AI trained on data from hospitals that underdiagnose certain populations |
| Representation bias | Some groups underrepresented in training data | Facial recognition less accurate on darker skin tones |
| Label bias | Human labelers apply inconsistent or biased labels | Content moderation model trained on biased human judgments |
Algorithmic Bias
The model architecture or training process amplifies biases in the data.
- Feedback loops: Model outputs influence future training data, reinforcing initial biases
- Optimization target bias: Model optimizes for a metric that correlates with a protected attribute
- Proxy discrimination: Model uses non-protected features that correlate with protected attributes
Deployment Bias
The model is used in a context or population different from what it was designed for.
- Model trained on US English applied globally
- Model trained on adult data used for decisions about minors
- Model trained on one industry vertical applied to another
Regulatory Landscape
| Regulation | Bias Requirements |
|---|---|
| EU AI Act | High-risk AI must be tested for bias, with documentation requirements |
| NYC Local Law 144 | Automated employment decision tools must undergo annual bias audits |
| Colorado SB 24-205 | Deployers of high-risk AI must conduct impact assessments including bias |
| EEOC Guidance | Employers liable for AI-driven hiring discrimination under Title VII |
| CFPB Guidance | Lenders must explain AI-driven adverse credit decisions, including bias factors |
| FDA AI/ML Guidance | Medical AI must demonstrate performance across demographic subgroups |
Bias Testing Framework
Pre-Deployment Testing
Step 1: Define protected attributes Identify which attributes are legally protected or ethically sensitive in your context: race, gender, age, disability, religion, national origin, sexual orientation, socioeconomic status.
Step 2: Disaggregated evaluation Run model evaluation benchmarks separately for each demographic subgroup. Compare performance metrics across groups.
Step 3: Fairness metrics
| Metric | What It Measures | When to Use |
|---|---|---|
| Demographic parity | Equal positive outcome rate across groups | When equal representation matters |
| Equalized odds | Equal true positive and false positive rates across groups | When error rates should be equal |
| Predictive parity | Equal precision across groups | When positive predictions should be equally reliable |
| Individual fairness | Similar individuals get similar outcomes | When case-by-case fairness matters |
No single metric captures all fairness concerns. Choose based on the specific use case and regulatory requirements.
Step 4: Intersectional analysis Test not just individual attributes but combinations (e.g., race × gender × age). Bias often emerges at intersections that single-attribute analysis misses.
Post-Deployment Monitoring
- Track outcome distributions across demographic groups over time
- Monitor for drift in fairness metrics
- Sample and review model decisions for bias indicators
- Collect user feedback segmented by demographics (where legally permissible)
Mitigation Strategies
| Strategy | Stage | What It Does |
|---|---|---|
| Data balancing | Pre-training | Adjust training data to improve representation |
| Data augmentation | Pre-training | Synthetically increase underrepresented examples |
| Bias-aware fine-tuning | Fine-tuning | Include fairness objectives in the training loss |
| Prompt engineering | Deployment | System prompt instructions to avoid biased outputs |
| Output calibration | Post-processing | Adjust output probabilities to equalize across groups |
| Human review | Deployment | Human oversight for high-stakes decisions |
| Red teaming for bias | Testing | Adversarial testing specifically targeting bias |
Documentation Requirements
For any AI system making decisions that affect people, document:
□ Intended use case and population
□ Training data sources and known limitations
□ Protected attributes considered
□ Fairness metrics evaluated and results
□ Identified biases and mitigation steps taken
□ Residual bias risks and compensating controls
□ Monitoring plan for ongoing bias detection
□ Review cadence and responsible team
Tools
| Tool | Purpose |
|---|---|
| AI Fairness 360 (IBM) | Open-source bias detection and mitigation toolkit |
| Fairlearn (Microsoft) | Fairness assessment and mitigation for Python |
| What-If Tool (Google) | Visual bias exploration for ML models |
| Aequitas | Open-source bias audit toolkit |
| SHAP / LIME | Model explainability — understand why the model makes biased decisions |