When AI Learns to Discriminate: New Research Shows LLMs Stereotype Job Candidates More Than Humans Do
New research from Princeton and the University of Chicago finds that LLMs like ChatGPT, Claude, and Gemini develop hiring biases faster and more severely than humans, scoring roughly 65% higher on a segregation scale. The study, reported by MIT Technology Review, shows that AI models learn to stereotype candidates by fictional ethnic group after just a few rounds of feedback — and that rewarding models for diverse hiring is more effective than simply instructing them to be fair.
The AI Gatekeeper Problem Nobody Is Talking About Loudly Enough
Before a single human hiring manager reads your résumé, an AI model may have already decided you are not worth their time. That scenario is no longer hypothetical — it is the present reality at thousands of companies deploying large language models (LLMs) to screen applications at scale. What makes this deeply unsettling is a new body of research suggesting that these AI systems do not simply mirror the biases humans baked into their training data. They can also manufacture entirely new biases on their own — and they do so more aggressively than humans.
According to reporting by MIT Technology Review (https://www.technologyreview.com/2026/07/20/1140655/ai-biases-hiring-humans/), researchers at Princeton University and the University of Chicago ran a revealing experiment using widely deployed models, including ChatGPT, Claude, and Gemini, to understand how quickly an AI forms stereotypes when it is given a hiring task and feedback on its choices.
The Experiment: A Simulated Hiring Game
The research team designed a hiring simulation adapted from an existing psychology study that originally examined how humans form stereotypes. Each AI model was framed as a consultant helping the mayor of a fictional city fill 20 different roles — from doctors and lawyers to child-care aides and janitors. Job candidates came from four fictional ethnic groups: Tufa, Aima, Reku, and Weki.
In every round, the model reviewed four candidates, one from each group, and selected whom to hire. After each hire, the model learned whether the chosen candidate succeeded. Here is the critical detail: all candidates were equally likely to succeed at every job, regardless of their group. The models had no legitimate statistical reason to prefer any group for any role.
Yet the models quickly began segregating candidates. When a model observed that an Aima candidate failed as a doctor — a role the model associated with high warmth and competence — it began systematically steering Aimas toward lower-status roles like janitor. A single data point became a group-wide verdict.
Scoring Worse Than Humans on Segregation
The results on the study’s segregation scale are striking. A score of 2 on the scale means every ethnic group has been completely confined to its own job niche — total segregation. Human participants in the original psychology study scored 0.84 on average. The AI models scored roughly 65% higher. OpenAI’s reasoning model o3 scored 1.83, approaching the theoretical maximum for segregation.
This is not a minor gap. It is the difference between a hiring manager who holds casual, correctable assumptions and a system that has already sorted every demographic into a rigid hierarchy after just a handful of observations. And because AI is increasingly doing the sorting at the top of the recruitment funnel — before any human sees the applications — the consequences are amplified at scale.
Why LLMs Stereotype So Readily
Ryan Liu, a PhD student at Princeton University and a coauthor of the study, which was presented at ICML in Seoul in July 2026, puts it plainly: LLMs “really are eager to create generalizations from limited data.” That eagerness is, in fact, a core feature of how they are built. Models are optimised on tasks in mathematics, coding, and science where generalising quickly from a few examples is precisely what earns a high score.
Psychologists describe the universal tension between sticking with what has worked before and exploring new options as the “exploration-exploitation dilemma” — the same instinct that makes you return to your favourite restaurant rather than risk a new one. LLMs resolve this dilemma by exploiting early patterns far too quickly in social contexts, locking in a generalisation before nearly enough evidence exists to justify it.
Notably, the newer and more capable reasoning models performed worse on bias, not better. OpenAI’s o3 and DeepSeek’s R1 — both marketed as advanced reasoning systems — showed stronger stereotyping tendencies than older models. Greater raw intelligence, in this framing, appears to mean greater confidence in flawed early conclusions.
Memory and Personalisation Make Things Worse
The findings land at a particularly sensitive moment. AI companies are in a race to build “agentic” models that remember user history across conversations and personalize responses over time. Angelina Wang, a computer scientist at Cornell University who was not involved in the study, explains that when a chatbot draws on its previous conversation history, it can “over-index on the same kinds of behaviors it’s experienced before” — building and reinforcing biases through accumulated experience.
Simply telling AI systems to remember less is not a viable fix. Users genuinely want their AI tools to remember context; it is one of the core selling points of modern assistants. As Wang notes, the field is still figuring out the right calibration — how much memory is useful without becoming a liability.
What Doesn’t Work — and What Might
The researchers tested whether simply instructing models to be fair would reduce their stereotyping. It did not produce meaningful change. Liu suggests that either the model cannot translate the instruction into altered behaviour, or the drive to optimise for successful hires overrides the fairness directive entirely.
Two interventions did show promise, however.
First, offering the models a bonus incentive for diverse hiring made them significantly less biased. This points to a design principle with real-world implications: if you want socially desirable behaviour from an AI, you need to build that objective directly into its reward structure, not rely on a vague instruction to “be fair.” As Liu puts it, the goal must “incorporate desirable social values in order to make the large language model act in socially desirable ways.”
Second, providing richer, relevant individual information about each candidate reduced segregation. In a parallel experiment within the same study, models were asked to resettle members of fictional ethnic groups across cities in Canada. When models received personally relevant data — such as age and education level — they relied less on ethnic group as a sorting signal. When they received irrelevant details like hair colour and tattoo shape, they fell back on ethnicity as the primary filter. The lesson: superficial data pollution pushes AI systems toward demographic shortcuts, while substantive individual context can counteract that tendency.
The Real-World Stakes for Job Seekers
For anyone applying for a role in India or globally, the implications are immediate and practical. Companies are already deploying LLMs not just to rank CVs but to conduct early-round interviews. If these systems generate their own biases from feedback loops — deciding, for instance, that candidates with certain names or from certain regions tend not to succeed, based on a handful of early outcomes — those biases will compound invisibly at the top of every hiring funnel.
The feedback loop in real hiring is slower than in the experiment; a company may take months or years to evaluate whether a new hire is performing well. But Liu notes that when feedback does eventually arrive, a model could still draw excessive conclusions from limited signals and project them forward onto future candidates.
“These novel biases — they’re sort of ever present,” says Liu. They are not inherited from historical human prejudice. They are generated fresh, by the model itself, from the data it encounters on the job.
What Organisations Deploying AI in HR Should Do Now
For HR leaders and procurement teams evaluating AI hiring tools, this research offers several grounded takeaways.
- Audit feedback loops. If your AI hiring tool updates its behaviour based on hiring outcomes, understand exactly how those updates work and what signals are feeding in.
- Demand diverse-hire incentives. Ensure that any model used in recruitment has explicit, measurable objectives around demographic balance, not just accuracy of hires.
- Prioritise rich candidate data. Systems that see more relevant individual information about candidates appear less likely to default to group-level stereotyping.
- Test for emergent bias regularly. The biases described in this research are not pre-baked into models at launch; they develop over time. Periodic auditing is not optional.
As LLMs increasingly influence who gets hired, who receives a loan, and who is granted parole, the bias landscape has expanded beyond what any fairness checklist drafted in 2020 could anticipate. The novel biases that AI generates from its own experience may ultimately prove harder to detect and correct than the historical biases researchers have spent decades cataloguing — precisely because no human ever taught them.
