AI Can Invent Its Own Hiring Biases, Study Finds

Companies are leaning on artificial‑intelligence systems to sift through résumés, screen video interviews and even hand out job offers. The promise is speed and objectivity, but a fresh study suggests the promise may come with an unexpected hitch: AI can conjure its own stereotypes.

In the experiment, a team led by researchers from Princeton University and the University of Chicago asked 15 different large language models (LLMs) to play a simple hiring game that had already been run with human volunteers. The game presented “candidates” – each identical in qualifications – but assigned them to one of four invented ethnic groups called the Tufa, Aima, Reku or Weki. After each hiring decision the model received feedback indicating whether the hire succeeded or failed.

Human participants quickly learned to favor or avoid certain groups based on the feedback they got, a phenomenon the authors call “bias formation.” What surprised the scientists was that the AI models showed the same tendency – and at a far higher rate. Some models repeatedly rejected candidates from a particular fictitious group after a single negative outcome, even though the groups were statistically indistinguishable.

“LLMs can spontaneously develop novel social biases about artificial demographic groups even when no inherent differences exist,” the authors wrote. In plain English: these systems are not just mirrors reflecting the prejudices we already know about; they can manufacture new ones from the patterns they observe.

The root of the problem, the researchers argue, lies in a decision‑making shortcut known as the explore‑exploit trade‑off. Humans constantly weigh whether to try something new (explore) or stick with what has worked before (exploit). AI, especially when trained to maximize a reward signal, leans heavily toward exploitation. It sees a pattern – “candidates like X tended to succeed” – and doubles down, ignoring the possibility that the pattern might be a fluke.

When the team compared models, OpenAI’s newest “o3” reasoning engine was the most severe in its stereotyping, followed closely by newer, larger versions from Anthropic, Google, Meta, DeepSeek and Alibaba. The researchers note that the more capable a model is at drawing precise inferences, the more likely it is to over‑fit to past outcomes, reducing its willingness to explore alternatives.

That matters because, according to a ManpowerGroup survey, more than nine out of ten firms now use AI somewhere in their talent‑acquisition workflow. Job seekers have already started to feel the sting. Workday, a major provider of HR software, is facing a class‑action suit alleging that its AI‑driven hiring tools discriminate against certain applicants. Similar claims have emerged at Meta, where employees say an internal AI system skewed layoff decisions against people with disabilities or those who took protected leave.

The ripple effects extend beyond hiring. AI‑generated biases have been reported in healthcare triage tools, loan‑approval algorithms and tenant‑screening platforms. In each case, a system that quickly spots patterns ends up generalizing in ways that can harm real people.

What’s the way forward? The authors suggest we need “targeted interventions that discourage harmful pattern‑matching while preserving the constructive abstraction that makes LLMs useful.” In practice that could mean injecting randomness, penalizing over‑reliance on past successes, or designing feedback loops that explicitly reward diversity of choice.

It’s a tall order, but as AI continues to seep into everyday decision‑making, finding that balance may be the most important challenge we face – not just for fair hiring, but for a society that wants technology to uplift rather than pigeonhole.