Measuring Bias in Job Recommender Systems: Auditing the Algorithms

We audit the job recommender algorithms used by four Chinese job boards by creating fictitious applicant profiles that differ only in their gender. Jobs recommended uniquely to the male and female profiles in a pair differ modestly in their observed characteristics, with female jobs advertising lower wages, requesting less experience, and coming from smaller firms. Much larger differences are observed in these ads’ language, however, with women’s jobs containing 0.58 standard deviations more stereotypically female content than men’s. Using our experimental design, we can conclude that these gender gaps are generated primarily by content-based matching algorithms that use the worker’s declared gender as a direct input. Action-based processes like item-based collaborative filtering and recruiters’ reactions to workers’ resumes contribute little to these gaps.