Bias Audit of HR Screening AI

PeopleFirst HR's AI screening system was meant to improve hiring efficiency. Instead, it became a discrimination engine. Women were 40% less likely to be recommended for interviews. Applicants from Romania, Bulgaria, and other countries were systematically downranked. The system had learned biases from historical hiring patterns, which themselves reflected discrimination.

Audit Methodology

We started with demographic parity analysis. We split the applicant pool by gender, nationality, age, and other demographic variables, then checked if the AI model recommended interviews at similar rates across groups. It didn't. Women were recommended at 35% vs. 58% for men. Eastern Europeans at 22% vs. 51% for Western Europeans.

Then we did feature importance analysis. Which features drove these differences? University choice (maybe women attended different universities). Resume gaps (maybe women had childbearing gaps). Spelling patterns (Eastern European names have distinct patterns). None of these features explicitly referenced gender or nationality, but the model learned to use them as proxies.

Remediation

We rebuilt the model with three changes: (1) Remove obvious proxy features (university, spelling patterns). (2) Add fairness constraints during training that penalized the model for demographic disparities. (3) Regular bias audits to catch new biases as hiring patterns change.

The retrained model achieved near-perfect demographic parity (≤2% difference in recommendation rates across groups) while maintaining 92% of the original model's predictive accuracy on "hire/no-hire" outcomes. Small loss in accuracy for massive fairness gain—the math is worth it.

Governance

We also helped PeopleFirst establish an AI Governance Council that reviews model performance quarterly, audits for bias, and manages documentation for GDPR compliance. This reduces the chance that future models develop hidden biases.