26% of Black applicants and 15% of Asian applicants applied to positions where the AI system discriminated against their racial group. That's 40,000 applications that would have advanced if the AI recommended them at the same rate as the most-favored group - typically white applicants. These numbers come from a new Stanford HAI study tracking 4 million real job applications across 1,700 postings at 150 employers, all screened by a single third-party AI vendor.
How One AI Vendor's Black Box Biases at Scale
3.4 million people submitted those applications across 11 industry sectors. The researchers applied the EEOC's four-fifths rule, which flags a position when one group is recommended at less than 80% of the rate of the most-recommended group. Pool all the vendor's recommendations together, and the adverse impact disappears. Look at each position separately - as employment law demands - and the discrimination surfaces. For example, the AI might recommend Black applicants for warehouse jobs but rarely for finance jobs, averaging out to a false clean bill of health.
Algorithmic Monoculture Creates Systemic Rejection
The study's second headline finding goes beyond bias per position. When many employers rely on the same vendor's recommendations, applicants face a systemic rejection risk. People who submit multiple applications to positions screened by that vendor are more likely to be rejected from every position they apply to than if each company decided independently. 10% of applicants who submit four applications are rejected from all of them. Contrast that with a prior study of 83,000 applications to 108 Fortune 500 firms (where AI use wasn't tracked): there, the rejection rate matched statistical independence. Market concentration in hiring AI matters.
Why This Matters Now
90% of U.S. employers use AI screening tools, most from the same few vendors. These tools are pervasively adopted, highly consequential, and opaque to the public. The Stanford study demonstrates that without independent research and per-position audits, bias and systemic rejection stay hidden. The key lesson: evidence-based AI policy can't exist without researchers getting inside the black box. This space is evolving fast as new language-model-based tools enter the market. The next wave of hiring AI will demand even more scrutiny.
Source: AI Hiring Tools Yield Racial Bias and Systemic Rejection; 26% Black & 15% Asian
Domain: hai.stanford.edu
Comments load interactively on the live page.