Home When the judge is a language model 00%

Essay · 16 September 2026

When the Judge Is a Language Model

Language models now read applications and return a number. This essay collects what the published research says about that number, examines the fourteen safeguards that deployers and regulators rely on, and asks why the same question is arriving in admissions, credit, insurance, public benefits, peer review and the courts.

Maksim Silchenko  ·  National University of Singapore

Documented failures
13each with a published study behind it
Safeguards in use
14none removes the failure it targets
Bias mitigations tested
15 to 20percent of those proposed in the hiring literature
Summary → Start reading Sources

Summary

Eight points

  1. A language model that screens applications returns a number about a person, and the studies that examined that number found it depends on things other than the file. The model version, the run, the order in which files are shown and the wording of the job description each move it. The failures below are documented in published work on real applications and on controlled test sets, not in one paper.1
  2. Removing names does not remove demographics. In Singapore, ethnicity was recovered from the language lines of a CV alone at a macro F1 of 0.97, and gender from the activity, volunteering and hobby lines at 0.98.1 Blinding 1,373 real teaching applications barely moved the .2
  3. The written rationale does not report the driver. With realistic company context, race and gender shifted interview decisions by up to 12 percent while the model's explanations never mentioned either.4
  4. The same file does not get the same number. One unchanged resume scored 50.5, 69.4 and 72.8 with three frontier models;8 13.6 percent of pairwise judgments flipped on an identical rerun;9 the first-listed of two CVs won 63.5 percent of comparisons across 22 models.10
  5. The human standard the model is checked against is itself noisy. Admissions officers agreed on personal qualities at a of 0.15 to 0.60,12 and outcomes exist only for the people who were admitted.14 Accuracy against the committee therefore has a ceiling, and a high score can mean the model has learned one rater's habits.
  6. Fourteen safeguards are in use, and none removes the failure it targets. Anonymisation and anti-bias instructions stop working on realistic input. Rubrics, juries, format choice and re-audits reduce one failure each and were mostly tested on synthetic input or general judging tasks. For six of the thirteen failures no safeguard with a positive published test exists, and for two there is no safeguard at all.
  7. Regulation creates the duty to report but does not supply the measurement. Of 391 employers covered by New York City's bias-audit law, 18 posted an audit;23 the enforcing agency found one non-compliance among 32 companies where the state auditor found at least 17.41 The EU AI Act classes recruitment, admission, credit, insurance, benefits and judicial assistance as high-risk, and in July 2026 the obligations for those systems were postponed to December 2027;4258 management standards certify that a process exists, not that it caught anything.4546
  8. The same instrument is being introduced beyond hiring, and its failure modes travel with it. Essay scoring, mortgage underwriting, peer review, insurance and public benefits already have documented cases or precedents, and several courts have met it. Because a shared model repeats one error at every gate at once, the time to require measurement is before the deployments become the default.

Every number on this page is tied to a source in the list at the end; every source was fetched and read before it was cited, and the bibliographic records were checked by machine against the project's bibliography on 15 and 16 September 2026. The essay describes what is known; it does not propose a method.

Part one

How a screener is used today

The setting is an admissions office or a recruiting team with more applications than readers. The model is asked to do what a first reader did: read the file and say how strong it is.

A typical deployment is simple to describe. An application arrives as one or more documents: a CV, a statement, transcripts, references. The documents are converted to text. Sometimes the name, the photograph and the dates are removed first. The text is placed in a prompt together with the job description or the programme's criteria, often with a rubric, and the on a fixed scale, a rank among the candidates, or a shortlist decision, usually with a written justification. The number goes to a person or a committee, who may read the justification, and the person decides. In the largest deployments one vendor's model reads the applications of many employers at once.19

The tool is described by its vendors as fast, consistent and free of the fatigue and prejudice of a human reader. Those are three separate claims, and each has now been tested. The evidence comes from audits of deployed systems, from controlled experiments that pass real applications through real models, and from the wider literature on language models used as judges. One end-to-end audit of a public employment agency covered about 497,000 candidate-vacancy entries over five years and seven pipeline stages, which gives a sense of the scale at which such systems already operate.39

The path from a file to a decision has five places where something can go wrong, and the thirteen failures in the next part are arranged along it. The diagram below follows that path; the failures appear where the studies located them.

1The fileCV, statement 2Preparationnames removed 3The modelprompt, rubric,one run,one version 4The numberscore or rank 5The committeereads, decides 6The outcomeknown for admits only ONE MODEL, EVERY EMPLOYER APPLICANTS ADAPT TO THE SCREENER THE YARDSTICK 1 2 3 4 5 6 7 8 9 10 11 12 13

  1. The path

    Six stages, two loops. The dashed loop at the bottom is the applicant learning what the screener rewards. The dashed box around the model is the same model serving every employer that buys it.

  2. A. The model sees what it should not

    Demographics are inferred from the lines that remain after the name is removed, the written rationale does not report the driver, and the direction of the bias changes with the model version.

    Problems 1 to 3

  3. B. The number is not reliable

    The same file gets different scores from different models and from the same model on a rerun, the order of the files changes the answer, and small differences at the margin are missed.

    Problems 4 to 7

  4. C. The yardstick is itself noisy

    The committee the model is checked against disagrees with itself, outcomes exist only for the admitted, and outcome labels carry the labour market's history.

    Problems 8 to 10

  5. D. People adapt to the screener

    Hidden instructions, fabricated credentials and machine-written statements change the file once the applicant knows a model will read it.

    Problem 11

  6. E. System and governance

    One shared model repeats its error at every gate at once, and nobody has established what the committee does with the number.

    Problems 12 and 13

Part two

Thirteen documented failures

Each failure below is a published finding, with the strongest evidence found for it. The numbers are the studies' own; the essay adds nothing to them.

AThe model sees what it should not

Problem 1 of 13

Demographics are inferred from proxies, even with the name removed

Ethnicity from language lines
0.97macro F1 across 18 models; 0.13 with those lines deleted
Gender from activities
0.98macro F1 from the volunteering and hobby lines
Gender from embeddings
0.996area under the curve, 1,112 real master's applications

The standard defence against demographic bias is to remove the direct identifiers: the name, the photograph, the address, sometimes the dates. The evidence is that the model rebuilds the profile from what remains. In a Singapore study, researchers took 100 real job descriptions, produced 4,100 variants of real CVs with the names and explicit demographic fields removed, and passed them through 18 language models. From the language lines alone, the models recovered the applicant's ethnicity at a of 0.97; with those lines deleted the figure fell to 0.13. From the activity, volunteering and hobby lines they recovered gender at 0.98. The inference reached the scores: across the 18 models, profiles read as Chinese male ranked highest and profiles read as Indian or Malay female ranked lowest.1

The result is not specific to one country. Blinding 1,373 real teaching applications barely changed the adverse-impact ratios, because the model inferred the demographics from the rest of the file.2 Text embeddings of 1,112 real master's applications predicted the applicant's gender with an of 0.996.3

Why the proxies are dense

In Singapore, language and school names correlate with ethnicity and nationality, the dates of national service give a man's age, and the tier of the university carries socioeconomic class. The lines that applicants are advised to include in order to appear well rounded are the lines that rebuild the profile. Demographic information is hard to hide from a system trained on this much human writing, and a screener that never sees a name can still rank by ethnicity.

Figure 1 Recovering ethnicity and gender from anonymised CVs
Macro F1 is the mean of the per-class F1 scores, from 0 to 1. Language lines carried the ethnicity signal almost on their own. Sources: Tan et al. 2026; Isley, Gaebler and Goel 2026.

Problem 2 of 13

The written rationale does not report the driver

Shift in interview decisions
up to 12%from race and gender, with realistic company context
Mentions of race or gender
0in the written explanations of those decisions
Hint revealed in the reasoning
under 20%of the cases in which a reasoning model used it, in most settings

A model can be asked to explain its score, and it writes a professional rationale. The question is whether the rationale reports what moved the number. In a study that added realistic company context to the prompts, such as an employer name and notes on culture fit, the applicant's race and gender shifted interview decisions by up to 12 percent, and the explanations never mentioned either. Anti-bias instructions in the prompt, which work in simplified settings, failed once the context was realistic; the one intervention that held across settings acted on the model's internal activations, which a closed API does not expose.4

Two further findings bear on this. In a mortgage-underwriting experiment with instruction-tuned models, the text output showed no bias while the internal representation still encoded the applicant's race, and re-injecting that representation into the model's processing reversed the decisions.50 In a study of reasoning models, the reasoning text mentioned the hint the model had actually used in fewer than 20 percent of the relevant cases in most of the settings tested.51

Alignment training and the gap between the words and the decision

does not close the gap. A study of dialect prejudice in Nature found that models trained in this way express fewer overt stereotypes while their covert stereotypes about speakers of African American English remain, so the training taught the models to conceal a prejudice rather than to remove it.49 Reading the rationale is therefore not an audit of the decision. It is a second output of the same model.

Problem 3 of 13

The direction of the bias changes with the model version

2023 model
+2.12points, the pro-White callback gap of the field experiments
Every model from 2024
null or reverseda reversal of up to 3.01 points the other way
After one round of alignment
+152%in the penalty against applicants with disabilities

The direction and the size of the bias depend on the release. GPT-3.5 rated names associated with White applicants above Asian, Black and Hispanic names.5 A 14-model audit that reproduced the paired-resume design of the labour-market field experiments found that the 2023 model showed the pro-White callback gap of 2.12 points, while every model released from 2024 showed a null gap or a reversal of up to 3.01 points in the other direction.6 In another study, one round of alignment training raised the advantage for female and Black applicants about fourfold and raised the penalty against applicants with disabilities by 152 percent.7

An audit therefore describes one release. It stops applying when the vendor updates the model, and vendors update without notice. Pushing one bias down can raise another.

Figure 2 The sign of the callback gap, by release year of the model
Schematic of the audit's finding: the 2023 model reproduced the human callback gap; later releases showed no gap or a reversal. Individual model values other than the two extremes are not reported here. Source: Gao, Jiang and Yan 2026.
BThe number is not reliable

Problem 4 of 13

The same CV receives different scores

One resume, three models
50.5 / 69.4 / 72.8on a 100-point scale
Shorter job description
+28.9%in one model's scores for the same applicant

If the score is a measurement, the same text should produce the same number. One unchanged resume was scored 50.5, 69.4 and 72.8 by three frontier models. Shortening the job description in the prompt inflated one model's scores for the same applicant by 28.9 percent.8

A swing of 22 points between vendors covers the distance between a rejection and a strong shortlist. Until its variance is known, the score is not a measurement in the ordinary sense of the word.

Figure 3 One unchanged resume, three frontier models
The same text received three scores 22 points apart on a 100-point scale. Source: Varshney and Ganuthula 2025.

Problem 5 of 13

The same model does not agree with itself

Pairwise judgments that flip on a rerun
13.6%above 20 percent on 28 percent of the questions; 56 percent on the worst
Identical requests at temperature zero
1,000 → 80distinct completions
Single-run rankings with an inverted pair
83%of the slices tested

A single model does not agree with itself. On identical reruns, 13.6 percent of pairwise judgments flipped; on 28 percent of the questions the flip rate exceeded 20 percent, and on the worst item it reached 56 percent.9 In single-run rankings, where a model orders a batch from best to worst, at least one pairwise rank was inverted in 83 percent of the slices tested.52

Setting the does not remove this. When 1,000 identical requests were sent to a model at temperature zero, the outputs fell into 80 distinct completions. The cause is not randomness in the sampling but the serving system: floating-point addition is not associative, the order of the additions changes with the size of the batch a request happens to share, and a small difference early in the computation can change the next token.33 The same non-determinism has been documented across providers and settings,53 and the author has observed it on a university computing cluster.

What this means for a screener

A file submitted on Tuesday and again on Wednesday can receive two different scores from the same model with the same prompt, and a rank order produced in one run can differ from the rank order of the next. A screener's score therefore has a run-to-run variance that has to be measured before the score is compared with anything, and averaging over reruns reduces the noise without touching the bias.

Figure 4 Rerun the judge: 200 identical pairwise judgments Simulation at the published rate
Reruns
0
Flipped this rerun
0 of 200
Judgments that flipped at least once
0 of 200
Simulation. Each dot flips independently at the published average rate of 13.6 percent per rerun; on the worst items in the study the rate was 56 percent. Source: Yagubyan 2026.

Problem 6 of 13

Order and format change the answer

First-listed CV wins
63.5%of pairwise comparisons across 22 models
Distance from the fair ideal
0.70 vs 0.20scoring one CV at a time against choosing between two
Adverse-impact ratio
changes signbetween score thresholds 4 and 5

Where a file sits in the stack changes the answer. Across 22 models, including reasoning models, the first-listed of two CVs won the comparison 63.5 percent of the time.10 The effect interacts with the quality of the CVs being compared.54

The matters as much. When one CV was copied eight times with a single marker of ethnicity or gender changed, a fair model would treat every copy alike. Asked to choose between the original and a marked copy, the models sat 0.20 from the fair ideal of one half; asked to score each copy alone from 0 to 100, they sat 0.70 from the ideal, 3.5 times further. The demographic gap was similar in both formats, at 6 to 7 percent; the difference came from the models giving different scores to identical CVs when judging them one at a time. Asking for a written rationale made the fairness of the scores worse.1 In the teaching-application audit, the adverse-impact ratio changed sign between score thresholds 4 and 5, so the same model could pass or fail a depending on where the cut was placed.2

Figure 5 Position and format
Left: the first-position advantage across 22 models (Rozado 2025). Right: the distance from the fair ideal under two question formats for the same CVs (Tan et al. 2026).

Problem 7 of 13

Small differences are missed, and the model abstains

Claude Sonnet 4
0.87 to 0.93accuracy on strict-dominance pairs
Llama 3.3 70B
0.44 to 0.50accuracy on the same pairs
Errors that were abstentions
93%unjustified refusals to choose

Real decisions are made at the margin between similar files, and this is where the models are weakest. Researchers built the other by a single attribute, such as one more year of relevant experience. Accuracy on those pairs was 0.87 to 0.93 for Claude Sonnet 4 and 0.44 to 0.50 for Llama 3.3 70B, and 93 percent of the errors were unjustified abstentions: the model declined to choose.11

CThe yardstick is itself noisy

Problem 8 of 13

Human raters disagree with each other

Agreement of admissions officers
0.15 to 0.60Krippendorff's alpha on personal qualities
Own-gender leniency
1.74logits, across 3,875 residency files and 69 raters
Model judges reproducing the minority expert reading
0 of 26on contested bar-examination essays

The model is called wrong when it disagrees with the committee, which treats the committee's label as the standard. Admissions officers coding the personal qualities of applicants agreed with each other at a between 0.15 and 0.60, where 1.0 is perfect agreement and 0 is chance.12 Across 3,875 residency files and 69 raters, raters were more lenient toward applicants of their own gender, a shift of 1.74 .13 On contested bar-examination essays where the examiners held a majority and a minority reading, none of 26 language-model judges reproduced the minority reading.55

Accuracy against the committee therefore has a ceiling set by the committee's own agreement, and a model that matches the committee closely may be matching one rater's habits rather than the applicant's qualities.

Figure 6 How much the human standard agrees with itself
Krippendorff's alpha for admissions officers' codes of personal qualities, 0.15 to 0.60. A rule of thumb treats 0.67 as the floor for tentative conclusions and 0.80 for reliable ones. Source: Lira et al. 2023.

Problem 9 of 13

Outcomes exist only for the admitted

Admitted in the example
28 of 100when two scores from 1 to 10 must add to at least 14
Correlation among the admitted
−0.50between two scores that are unrelated in the pool

A screener cannot be checked on the applicants it would have admitted but the committee rejected, because they have no outcome (the problem); naive corrections can make a model appear to beat the human readers when it does not.14 The distortion, known as , has a simple form. Suppose the committee scores a test and work experience, each from 1 to 10, and the two are unrelated in the applicant pool. If it admits anyone whose two scores add to at least 14, it admits 28 of every 100 applicants, and among those admitted the correlation between test and experience is exactly negative 0.50: to get in with a modest test score you needed strong experience, and the reverse. Any check of whether experience predicts success that is run only on admits inherits this, and the same selection can shrink a real effect to zero or reverse its sign.

Figure 7 Selection on the sum of two independent scores Worked example
Admitted
28 of 100
Correlation in the pool
0.00
Correlation among the admitted
−0.50
Worked example. Two independent scores, each uniform from 1 to 10. Selecting on their sum induces a negative correlation among the admitted at every threshold; selecting on one score alone does not. The arithmetic is exact, not simulated.

Problem 10 of 13

Outcome labels import the market's history

Trained on past hires
9.4% → 4.2%share of Black and Hispanic applicants interviewed
Exploration-based model
24.3%the same share

Training on outcomes imports the history of the market that produced them. A hiring model trained on past successful hires cut the share of Black and Hispanic applicants receiving interviews from 9.4 to 4.2 percent; an exploration-based model that valued information about candidates raised it to 24.3 percent.15 Salary and seniority several years after a decision depend on what happened after the decision, such as mentoring, promotion practices and household choices, so they carry the biases of the labour market before they can serve as a target for the screener.

DPeople adapt to the screener

Problem 11 of 13

The file changes once a model reads it

Personal statements flagged as AI-assisted
2.6% → 22.5%in one residency cycle, one year apart
Cover-letter quality and callbacks
−51%in the correlation once an AI writing tool was available
Advantage from matching the screener's model family
23 to 60%higher chance of being shortlisted

Once applicants know a model reads the file, the file changes. , instructions hidden in the document for the model to read, are present in a corpus of about 200,000 real resumes.16 In an experiment that ran a multi-stage hiring pipeline 180 times, the fully automated version produced at least one unsupported claim in 96.7 percent of its outputs, at 6.80 per output.17

The signal in the writing erodes as well. On a large freelance platform, the correlation between the quality of a cover letter and the chance of a callback fell by 51 percent when an AI writing tool became available.18 In one surgical residency cycle, the share of personal statements flagged as AI-assisted rose from 2.6 to 22.5 percent in a year.56 A screener built on the same model family as the applicant's writing tool gave those applicants a 23 to 60 percent higher chance of being shortlisted.57 The applicant presses a button to produce the text and the screener presses a button to produce the score.

Figure 8 Adaptation, in three measured quantities
Sources: Subillaga et al. 2025; Cui, Dias and Ye 2025; Xu, Li and Jiang 2025.
ESystem and governance

Problem 12 of 13

Everyone is rejected by the same model

Applications routed by one shared screener
14.7% / 25.9%to positions with adverse impact on Asian and on Black applicants
When two models both err
60%they give the same wrong answer, across more than 350 models

When every employer buys the same model (an ), an applicant who falls into its blind spot is rejected everywhere at once. One vendor's shared screener sent 14.7 percent and 25.9 percent of applications to positions with adverse impact on Asian and on Black applicants respectively.19 A shared model can lower overall welfare even when it is more accurate than the human readers it replaces, because its errors are synchronised.20 Across more than 350 models, when two models both erred they gave the same wrong answer 60 percent of the time, and larger, more accurate models were more correlated, not less.31 A human hiring market has diversity of judgment: one reader's rejection is another's interview. A shared screener does not.

Figure 9 One model, many gates
Sources: Bommasani et al. 2026; Kim, Garg, Peng and Garg 2025.

Problem 13 of 13

Nobody knows how the committee will use the number

Reviewers shown a favourable decile
0.42percentage points of movement, standard error 0.33, almost 20,000 applicants
Employers that posted a bias audit
18 of 391covered by New York City's law

The number goes to a person, and it has not been established what the person does with it. In a field experiment with almost 20,000 applicants to a selective college, experienced reviewers shown a favourable algorithmic decile moved their own judgments by 0.42 percentage points, with a standard error of 0.33: they did not use it.21 In laboratory studies, biased recommendations moved screeners.22 The two results are not contradictory; they show that the effect of the number depends on who receives it and how, and that this has to be measured in each deployment. Of 391 employers covered by New York City's bias-audit law, 18 posted an audit.23

Taking stock

The screener infers the applicant's demographics from the hobbies and the languages on the file. Its written rationale does not report why it scored the file as it did. Its biases change with each release. It gives a different score depending on which other requests shared the server at that moment, and a different one again depending on which file it read first. It abstains where a decision requires nuance. It is checked against a committee that agrees with itself at 0.15 to 0.60, on the applicants that committee admitted. And once it is in place, the applicants write for it.

None of these findings depends on one paper. Each is supported by at least one study on real applications or on a controlled test set, and several have been replicated across model families and countries. The next part asks what deployers do about them.

Part three

The safeguards in use, and what the tests say

Fourteen measures appear in vendor material, in regulation and in the research literature. A measure counts as working here only where a published test shows it reducing a measured failure on realistic input.

The matrix below places each measure against the failures it is meant to address. A teal cell marks a published test in which the measure reduced the failure; an ochre cell marks a measure that is untested on a screener, tested only on synthetic input, or supported by mixed results; a rust cell marks published evidence that the measure fails on realistic input. A blank cell means the measure does not address that failure. The hatched columns are failures for which no measure was found at all. Selecting a coloured cell shows the evidence on both sides.

Figure 10 Fourteen safeguards against thirteen failures
a published test shows a reductionuntested on a screener, or mixedfails on realistic inputno safeguard found
Reading down a column shows what protects against one failure; reading across a row shows what one measure is asked to do. Six of the thirteen columns contain no teal cell. Sources: the fourteen entries below and the studies in Part two.

A. On the input side

  1. Remove names, photographs and dates. The most common measure and the one regulators expect; it removes the direct signal. Against it: the model rebuilds the profile from what remains. Blinding 1,373 real teaching applications barely moved the adverse-impact ratios,2 and in Singapore ethnicity was recoverable from the language lines alone.1 Necessary, not sufficient.
  2. Structured input: fixed fields, fixed section order, length caps. In principle this removes at the source the variation that moves scores, since the first-listed CV wins 63.5 percent of comparisons10 and a shorter job description inflated one model's scores by 28.9 percent.8 Against it: no published test of a fixed-form input on a screener was found. The evidence is of the problem, not of the fix; applicants adapt to whatever the form rewards; and the demographic channel is untouched.
  3. Injection and hidden-text detection. Hidden prompts in structured documents can be detected on principle,24 and injections are present in a corpus of about 200,000 real resumes, so detection has a target.16 Against it: it must run permanently, and it does not cover the pipeline's own fabrications. A fully automated multi-stage pipeline produced an unsupported claim in 96.7 percent of outputs; prompt guardrails cut the density by 86 percent and human checkpoints reduced it further, neither to zero.17

B. On the model and prompt side

  1. Anti-bias instructions in the prompt. Low in cost, applicable to any model, effective in controlled evaluations. Against it: the instructions stop working once realistic context, such as an employer name or culture-fit notes, is added. The only intervention that held across settings acted on the model's activations, which a closed API does not expose.4
  2. , where the model extracts features and a fixed rule decides. The strongest positive evidence in the literature: decomposition cut intervention flips by a median of 49 percent across 11 models;25 binary rubric criteria with filtering across three judge families removed all disagreement on 16 of 16 ranks over 1,000 real chats;26 across nine debiasing strategies the technique mattered more than the model tier.27 Against it: judges still favour their own model family under rubric criteria,28 which matters as more CVs are written with language models, and the rubric encodes the committee's own criteria, so the noise in the human standard is inherited rather than removed.
  3. Several models: a , or averaged scores. A panel of smaller judges from different families beat one large judge, reduced self-preference and cost about seven times less;29 averaging removes exactly the members' disagreement from the error.30 Against it: models err together. Across more than 350 models, two that both err agree 60 percent of the time, and larger models are more correlated;31 one systematically failing member gives the consensus an unbounded bias, and a larger panel does not fix it, only a robust aggregator does.32 Shared bias is what a monoculture spreads.19
  4. Temperature zero, repeated runs, deterministic serving. Batch-invariant kernels remove the hardware source of run-to-run noise,33 the temperature setting itself changes what a judge returns,34 and averaging reruns shrinks the noise at a known cost. Against it: temperature zero is not deterministic on standard serving, where 13.6 percent of pairwise judgments flip on an identical rerun.9 Reruns average noise, not bias, and no hiring-specific effect size for majority voting was found.
  5. Choice of format: pairwise or pointwise. The formats fail differently, so the choice is a lever: under demographic swaps the pairwise format sat 0.20 from the fair ideal against 0.70 for pointwise scoring.1 Against it: pairwise comparison amplifies wording and tone cues because the judge contrasts them directly,35 carries the first-position advantage, and is the format with the 13.6 percent rerun flips. Neither format is safe alone.
  6. Interventions inside the model: . The one measure that held across realistic settings in the study of realistic context;4 judge bias occupies a low-dimensional activation subspace, and steering it moves scores about ten times more than random directions.36 Against it: it needs the weights, which a school or a vendor using an API model does not have, and it is untested on real applications. This is research, not current practice.
  7. Calibration or fine-tuning on validated labels. Human labels can be budgeted to calibrate several judges at once,37 and a preregistered design re-anchors three model families against blinded human ratings every week.38 Against it: validity is rarely measured because it needs a labelled corpus the model has not seen;11 labels exist only for the admitted;14 and a model trained on past hires cut the Black and Hispanic interview share from 9.4 to 4.2 percent.15

C. Process and people

  1. Human review and escalation. Human checkpoints reduced fabrication beyond guardrails alone;17 in the one end-to-end audit of a deployed hiring pipeline, the human stages corrected some disparities;39 escalation rules with provable agreement guarantees exist for general judging tasks.40 Against it: the same human stages introduced disparities; experienced reviewers shown an algorithmic decile moved 0.42 percentage points;21 and reviewers agree with each other at an alpha of 0.15 to 0.60.12 Approval by a reviewer is not verification.
  2. Re-audit on every model version. The direction of the bias is a property of the release: in the 14-model audit the 2023 model reproduced the pro-White callback gap and every later model showed a null gap or a reversal,6 and alignment training amplified the role of race and gender in another.7 Against it: it is almost never done. Vendor updates are silent, statutory audits are annual and self-reported, and no off-the-shelf harness reruns the test at low cost.

D. Governance

  1. Statutory bias audits with (New York City Local Law 144). The law forces a measurement in a form regulators read: selection-rate ratios and the four-fifths rule. Against it: 18 of 391 covered employers posted an audit;23 the enforcing agency found one non-compliance among 32 companies where the state auditor found at least 17;41 and aggregate selection rates cannot see Problems 1, 2, 4 or 5.
  2. Certification and standards (vendor audits, the , , the EU AI Act, Singapore's ). The EU AI Act makes recruitment AI high-risk, with logging, human oversight and monitoring, obligations now due from December 2027;4258 Colorado enacted a duty-of-care and impact-assessment statute in 2024 and, before it took effect, replaced it in May 2026 with a narrower disclosure regime from January 2027;4360 AI Verify's fairness pillar tests decision consistency under attribute changes.44 Against it: the NIST framework and ISO/IEC 42001 certify that a process exists, not that it caught anything;4546 the most audited vendor's own earlier auditor described its review as not comprehensive;47 and in the hiring literature, 15 to 20 percent of the mitigations proposed for bias and discrimination report an empirical test, and about 60 to 65 percent of all mitigation mentions are proposed without one.48

Why the safeguards do not close the gap

Read together, the fourteen entries share a small number of reasons for falling short, and the reasons are structural rather than a matter of effort.

  1. They act where the failure is visible, not where it lives. Removing the name acts on the file; the demographic signal is in the hobbies and the languages. An instruction acts on the prompt; the decision is made in the activations. The one intervention that held in realistic settings needs the weights, which the deployer does not have.4
  2. They were tested somewhere else. Rubrics, juries and format choice were tested on synthetic input or on general judging tasks. The scoping review of the hiring literature found that 15 to 20 percent of the mitigations proposed for bias and discrimination report an empirical test, that about 60 to 65 percent of all mitigation mentions are proposed without one, and that fairness outcomes were measured about a third as often as efficiency.48 The frontier is not a shortage of proposals but the absence of tests on real screening input.
  3. They reduce noise, not bias. Reruns and juries shrink the run-to-run variance. They leave the shared component of the error in place, and the shared component is what a monoculture spreads: two models that both err agree 60 percent of the time.31
  4. They inherit the standard they are calibrated to. A rubric encodes the committee's criteria; a fine-tuned model learns the committee's labels; both inherit the committee's agreement of 0.15 to 0.60 and the selection of who has an outcome at all.1214
  5. They are done once, and the failure is per release. An audit describes the model that was audited. The sign of the callback gap changed between the 2023 release and every release since.6
  6. They measure the report, not the mechanism. A selection-rate ratio, a log and a human signature are the form the governance measures take. None of them can see a rerun flip, a position effect, or a rationale that does not report the driver.

Measures with positive published evidence exist for Problems 2, 3, 4, 5, 6 and 11, each partial, and for Problem 1 only inside the model. No measure with positive evidence was found for Problems 7, 8, 9, 10, 12 or 13, and for Problems 9 and 10 no measure was found at all. That is the state of the safeguards in September 2026, as far as the published record shows; a measure may exist in a vendor's practice without a published test, and the essay cannot speak to those.

Part four

What the rules require, and what they measure

Governance of automated screening exists in several jurisdictions. What it requires is a report in a fixed form. What it does not require is the measurement that found the failures in Part two.

The duty to report

New York City's has been enforced since 5 July 2023. An employer using an automated employment decision tool must obtain an independent bias audit at least annually, publish a summary, and notify candidates ten business days before the tool is used. The audit reports selection rates and impact ratios by sex, race and ethnicity and their intersections, in the four-fifths form that employment law has used for decades.59 In the first year, of 391 employers that researchers identified as covered, 18 posted an audit.23 The New York State Comptroller then audited the enforcing agency for July 2023 to June 2025: the agency had logged two complaints, reviewed the websites and audits of 32 companies and found one instance of non-compliance, while the Comptroller's auditors found at least 17 instances of potential non-compliance among the same 32, and 75 percent of test calls to the city's complaint line never reached the agency.41

Figure 11 The bias-audit law in its first two years
Left: 18 of 391 covered employers posted an audit (Wright et al. 2024). Right: the same 32 companies, reviewed by the enforcing agency and by the State Comptroller's auditors (Office of the New York State Comptroller 2025).

At the federal level, the Equal Employment Opportunity Commission's position has been that Title VII, the Americans with Disabilities Act and the Age Discrimination in Employment Act already reach automated tools through ,61 and its one settled case involved hard age cut-offs in an applicant system, resolved for 365,000 US dollars in 2023.62 Its 2023 technical assistance on assessing adverse impact in software and algorithms was removed from the agency's website on 27 January 2025, following an executive order of 23 January.63 The litigation that matters most is Mobley v. Workday, in which a federal court allowed disparate-impact claims against the vendor to proceed on the theory that the vendor acts as the employer's agent, in July 2024, and conditionally certified a nationwide collective of applicants aged 40 and over in May 2025; in 2026 the court held that the age statute covers applicants, treated the vendor's own bias-testing data as privileged while ordering other records produced, and let proxy-discrimination claims proceed. No trial date has been set.6465

Two states have moved further. Illinois amended its Human Rights Act, in force from 1 January 2026, to make it a civil rights violation for an employer to use artificial intelligence that has a discriminatory effect on a protected class, to bar the use of postal codes as a proxy, and to require notice; the implementing rules on that notice were proposed in May 2026 and withdrawn in June.66 Colorado signed a duty-of-care statute with annual impact assessments in May 2024, delayed it to June 2026, and then, in May 2026, repealed and re-enacted it as a narrower disclosure regime for automated decision-making technology from 1 January 2027. The original framework never took effect.4360

The high-risk category

The European Union's AI Act lists, in its , the uses it classes as high-risk. They include recruitment and the selection of natural persons, access and admission to educational institutions and the evaluation of learning outcomes, creditworthiness, risk assessment and pricing in life and health insurance, eligibility for essential public benefits, law enforcement, migration and border control, and systems "intended to be used by a judicial authority or on their behalf to assist a judicial authority in researching and interpreting facts and the law and in applying the law to a concrete set of facts".42 The list is, almost exactly, the list of places where a language model is now being asked to return a number about a person. The obligations that attach to the category, including logging, human oversight and post-market monitoring, were due to apply from 2 August 2026. On 27 July 2026 a further regulation entered into force that moved that date to 2 December 2027 for the Annex III systems and to 2 August 2028 for high-risk products, on the ground that the harmonised standards were not ready.58

Singapore's instruments are voluntary. AI Verify, launched in 2022 and open-sourced in June 2023, is a self-assessment toolkit whose fairness tests check whether a system makes the same decision when a sensitive attribute is changed;44 the Tripartite Guidelines on Fair Employment Practices operate on a comply-or-explain basis and do not mention automated tools;67 the Workplace Fairness Act, passed in January 2025, will be the first binding employment-discrimination statute to reach decisions assisted by such tools, and has not commenced.68 The United Kingdom's judiciary issued guidance for judicial office holders in December 2023, updated in April and October 2025, which permits some uses of the tools and reminds judges that they are "personally responsible for material which is produced in their name".69

Standards and certificates

The management standards that vendors cite are process standards. The NIST AI Risk Management Framework states that it "does not prescribe risk tolerance" and that its actions "do not constitute a checklist"; it structures an organisation's own process.45 ISO/IEC 42001 is described by the standards body as providing "a practical way of managing AI-related risks and opportunities across an organization" rather than "looking at the details of specific AI applications".46 A certificate under either says that the organisation has a process. It does not say that the process caught a rerun flip, a position effect or a demographic inference. The most audited vendor in the sector announced an external bias audit in 2023 in a release that described its previous review as not comprehensive.47

2023

New York City · 5 JulyLocal Law 144 enforced. Annual bias audits with impact ratios, a public summary and candidate notice.
United States · AugustiTutorGroup settles with the EEOC for 365,000 US dollars over age cut-offs in an applicant system.
Singapore · JuneAI Verify open-sourced. A voluntary testing toolkit; its fairness test changes a sensitive attribute and checks the decision.
United Kingdom · DecemberJudicial guidance on artificial intelligence issued. Judges may use the tools for some tasks and remain personally responsible.

2024

European UnionThe AI Act is adopted. Annex III classes recruitment, admission, credit, insurance, benefits and judicial assistance as high-risk.
Colorado · MaySB 24-205 signed. A duty of care and annual impact assessments for high-risk systems, to start in February 2026.
United States · JulyMobley v. Workday proceeds. Disparate-impact claims against the vendor go forward on the agent theory.

2025

United States · 27 JanuaryThe EEOC's adverse-impact guidance is removed from its website after an executive order of 23 January.
United States · 16 MayA nationwide collective is conditionally certified in Mobley v. Workday for applicants aged 40 and over.
Colorado · 28 AugustThe effective date is delayed to 30 June 2026.
European Commission · 19 NovemberThe Digital Omnibus is proposed, linking the high-risk deadline to the readiness of standards.
New York State · 2 DecemberThe Comptroller's audit is published. One non-compliance found by the agency; at least 17 by the auditors.

2026

Illinois · 1 JanuaryThe Human Rights Act amendment takes effect. Discriminatory effect from artificial intelligence becomes a civil rights violation; postal-code proxies are barred.
Colorado · 14 MaySB 26-189 repeals and re-enacts the statute as a disclosure regime for automated decision-making technology, from 1 January 2027.
United States · March to JuneThree further rulings in Mobley v. Workday. Applicants are covered; the vendor's own bias-testing data is privileged; proxy claims proceed.
European Union · 27 JulyRegulation (EU) 2026/1744 enters into force. The high-risk obligations move to 2 December 2027.

2027

Colorado · 1 JanuaryThe disclosure regime applies.
European Union · 2 DecemberThe Annex III obligations apply, if the date holds.
Singapore · expected by year endThe Workplace Fairness Act commences.

What the layer measures

Placed against Part two, the governance layer has a definite shape. It creates the obligation to measure and fixes the form of the report: a selection-rate ratio, a log, a human in the loop, a certificate that a process exists. It supplies no measurement that reaches the mechanisms. A selection-rate ratio computed once a year cannot see that the ratio changes sign between two score thresholds, that the same file scores differently on a rerun, that the first-listed file wins, or that the rationale does not report the driver. A log records what the model returned, not whether it would have returned it again. A human signature records that a person approved the output, and the field experiment in Part two shows how little the person may have engaged with it. And the deadlines that would have made the European obligations bite were moved back by sixteen months in the summer of 2026, before any Annex III system had been through them.

The state of the governance, in one paragraph

Every jurisdiction examined here has either a voluntary instrument, a statute that reports selection rates, or a statute whose obligations have been postponed or narrowed before taking effect. None requires the measurements that the published studies used to find the thirteen failures. Enforcement of the one operating statute found one non-compliance where an independent auditor found seventeen. The published record on the safeguards themselves shows 15 to 20 percent of the proposed bias mitigations tested. The gap between what is known about the instrument and what is required of it is the subject of the rest of this essay.

Part five

Beyond hiring: where the same instrument is heading

A screener is one use of a general technique: a language model reads a document about a person and returns a judgment. The technique is now being applied wherever such documents queue up.

The thirteen failures are properties of the technique, not of the hiring setting. A model that rebuilds ethnicity from the languages on a CV rebuilds it from the languages in a loan application. A rationale that does not report the driver of a hiring score does not report the driver of a claim denial. A rerun that flips 13.6 percent of pairwise judgments flips them whether the pair is two candidates or two essays. The list below is the Annex III list of the EU AI Act, in effect: the uses that a legislature has already identified as high-risk are the uses where the instrument is arriving. Each entry records only what has been documented, with a date; the diagram marks whether the domain has a documented deployment of a language model, a precedent from an earlier automated system, a published study of the failure modes, or a rule that names it.

Select a sector, or a card below, for the documented cases and sources. a documented deployment of a language model   a precedent from an earlier automated system   named as high-risk or regulated

Three things the cases have in common

The decision is about a named person, made from a document they wrote. The output is a number, a label or a draft that a person is supposed to check. And in every documented case where the checking was examined, the checking was thinner than the policy said: a claim reviewed in 1.2 seconds, an order signed with citations that did not exist, a review written by a model under a rule that required disclosure.

What travels

Each domain inherits the failures that its documents and its decision structure expose. Essay scoring exposes the proxy channel through first language and writing style,7172 the prestige channel through the name of the institution,73 and the adaptation loop through machine-written essays. Lending exposes the gap between the rationale and the driver, in a setting where the regulator has stated that "creditors must be able to specifically explain their reasons for denial. There is no special exemption for artificial intelligence."75 Peer review exposes prompt injection, since a hidden instruction in a manuscript is the same object as a hidden instruction in a CV,87 and the monoculture, since one model reviewing every submission repeats one preference across a whole field.89 Claims processing exposes the human-in-the-loop failure in its starkest documented form: 1.2 seconds of review per denial.76 Courts expose the rationale failure as fabricated citations in signed orders.84 And every domain in the list is one where the person judged has little means of knowing that a model was involved, and less of knowing what it saw.

The earlier systems in the benefits and immigration cases were not language models; they were risk scores built from a handful of fields, one of which was nationality. They are included because they show what happened when an automated score about a person was deployed at national scale with a thin human check and no measurement of its behaviour: a government resigned,79 a royal commission found a scheme "neither fair nor legal",80 and a tool was withdrawn on the eve of a court hearing.82 The language model differs from those systems in one respect that matters here: it reads the whole document, so the proxies it can use are not a handful of fields but everything the person wrote.

Part six

Why the timing matters

Four properties of the instrument make the order of events important: what is required of it now will be harder to require later.

A shared model repeats one error at every gate

The failures in Part two are properties of a model, and the same model is being bought by many institutions at once. In a human market, one reader's rejection is another reader's interview. When one screener serves every employer, an applicant in its blind spot is rejected everywhere, and the welfare loss can exceed the gain from the model being more accurate on average.20 Two models from different vendors are not independent readers either: when both err they agree 60 percent of the time, and the larger models are the more correlated.31 A measurement requirement introduced after a monoculture has formed applies to one model that has already become the default; introduced before, it shapes which models become the default.

The failures are per release, and releases are silent

The sign of the callback gap changed between the 2023 model and every model since,6 and one round of alignment training raised one penalty by 152 percent.7 A vendor updates the model without notice, and a statutory audit is annual and self-reported. Every year in which the rules do not require a per-release measurement is a year of decisions made under a bias whose direction nobody recorded.

The applicant pool adapts, and does not adapt back

The share of personal statements flagged as machine-assisted rose from 2.6 to 22.5 percent in one year;56 the correlation between cover-letter quality and callbacks fell by 51 percent when a writing tool became available;18 the writing tool and the screener favour each other.57 Each cycle of screening teaches the next cohort what the screener rewards. A standard introduced later meets a pool that has already been shaped, and the signal it would have measured is partly gone.

The rules being written now fix the form of the report for years

New York City fixed the four-fifths selection-rate ratio as the form of its audit in 2021, and every audit since has taken that form.59 The European harmonised standards that will define what "tested" means for a high-risk system are being drafted now, against a deadline that was moved to December 2027;58 Colorado's framework was narrowed to disclosure before it took effect;60 Singapore's binding statute has been passed and has not commenced.68 The technical content of these instruments, which measurements they require, at which cadence, on what input, is being decided in 2026 and 2027, and the published record shows that the measurements that find the failures are not the measurements the instruments currently name.

This essay does not argue that the tools should not be used, and it does not claim that the human readers they replace were reliable; Part two shows they were not. It argues something narrower: the instrument is being deployed before the measurements that would characterise it exist in the rules that govern it, and the moment to require those measurements is before the deployments become the default. Whether the requirement is proportionate in a given setting is a judgment the essay leaves to the people who make it.

Coda

What should we do

Research on this question is in progress in collaboration with the National University of Singapore. The results will be published in 2026 and 2027.

The numbers on this page are of three kinds: results reported in published studies, figures from official documents and court records, and one worked arithmetic example. Each is tied to a source in the list below, fetched and read before it was cited. Regulatory dates are as of 16 September 2026 and will change. Corrections are welcome by email.

Appendix

Sources

Ninety-four sources. Studies are listed with their arXiv identifier or DOI; documents and reports with the address at which they were read. Every source was fetched and read before it was cited; records 1 to 57 were checked by machine against the project bibliography on 15 September 2026, and records 58 to 94 were fetched on 16 September 2026. Hover over or select any superscript in the text to see its source.

Show the list
  1. Small Changes, Big Impact: Demographic Bias in LLM-Based Hiring Through Subtle Sociocultural Markers in Anonymised Resumes. Tan, B. C. Z., Khoo, S., Doan, B. N., Liu, Z. et al. arXiv, 2026. arXiv:2603.05189
  2. Auditing the Use of Language Models to Guide Hiring Decisions. Gaebler, J. D., Goel, S., Huq, A., Tambe, P. arXiv, 2024. arXiv:2404.03086
  3. Mitigating Label Bias with Interpretable Rubric Embeddings. Isley, C., Gaebler, J. D., Goel, S. arXiv, 2026. arXiv:2605.21455
  4. Robustly Improving LLM Fairness in Realistic Settings via Interpretability. Karvonen, A., Marks, S. D. arXiv, 2025. arXiv:2506.10922
  5. The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring. Armstrong, L., Liu, A., MacNeil, S., Metaxa, D. arXiv, 2024. arXiv:2405.04412
  6. Can LLMs Hire Fairly? Racial Bias in Resume Screening. Gao, Z., Jiang, W., Yan, Y. arXiv, 2026. arXiv:2606.28978
  7. AI Alignment Amplifies the Role of Race, Gender, and Disability in Hiring Decisions. Wang, Z., Shen, G., Thaler, M. arXiv, 2026. arXiv:2605.13866
  8. Signal or Noise? Evaluating Large Language Models in Resume Screening Across Contextual Variations and Human Expert Benchmarks. Varshney, A., Ganuthula, V. R. R. arXiv, 2025. arXiv:2507.08019
  9. The Coin Flip Judge? Reliability and Bias in LLM-as-a-Judge Evaluation. Yagubyan, A. arXiv, 2026. arXiv:2606.13685
  10. Gender and Positional Biases in LLM-Based Hiring Decisions: Evidence from Comparative CV/Résumé Evaluations. Rozado, D. arXiv, 2025. arXiv:2505.17049
  11. Measuring Validity in LLM-based Resume Screening. Castleman, J., Shen, Z., Metevier, B., Springer, M. et al. arXiv, 2026. arXiv:2602.18550
  12. Using artificial intelligence to assess personal qualities in college admissions. Lira, B., Gardner, M., Quirk, A., Stone, C. et al. Science Advances, 2023. doi:10.1126/sciadv.adg9405
  13. Examining Differential Rater Functioning and Bias in the Holistic Review of Residency Applications. Barnes, L. A., Sebok-Syer, S. S., Wind, S. A., Nord, K. M. Journal of Graduate Medical Education, 2026. doi:10.4300/jgme-d-25-00491.1
  14. The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables. Lakkaraju, H., Kleinberg, J., Leskovec, J., Ludwig, J., Mullainathan, S. Proceedings of the 23rd ACM SIGKDD Conference, 2017. doi:10.1145/3097983.3098066
  15. Hiring as Exploration. Li, D., Raymond, L., Bergman, P. arXiv, 2024. arXiv:2411.03616
  16. Measuring Real-World Prompt Injection Attacks in LLM-based Resume Screening. Zhang, M., Jia, Y., Tan, Z., Jiang, S. et al. arXiv, 2026. arXiv:2605.28999
  17. Mitigating Fabrication in Multi-Stage LLM Pipelines for Hiring: An Empirical Evaluation of Prompt Guardrails and Human-in-the-Loop Checkpoints. Takano, H. arXiv, 2026. arXiv:2608.26171
  18. Signaling in the Age of AI: Evidence from Cover Letters. Cui, J., Dias, G., Ye, J. arXiv, 2025. arXiv:2509.25054
  19. Algorithmic Monocultures in Hiring. Bommasani, R., Bana, S. H., Creel, K., Jurafsky, D. et al. arXiv, 2026. arXiv:2605.27371
  20. Algorithmic Monoculture and Social Welfare. Kleinberg, J., Raghavan, M. Proceedings of the National Academy of Sciences, 2021. doi:10.1073/pnas.2018340118
  21. Does Algorithmic Uncertainty Sway Human Experts? Evidence from a Field Experiment in Selective College Admissions. Lee, H., Alvero, A. J., Kizilcec, R. F., Joachims, T. arXiv, 2026. arXiv:2602.00241
  22. Resume Screening, Fast and Slow: (Biased) AI Recommendations' Influence on Human Decision Making. Wilson, K., Sim, M., Gueorguieva, A.-M., Chatterjee, S. et al. arXiv, 2026. arXiv:2606.22213
  23. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. Wright, L., Muenster, R. M., Vecchione, B., Qu, T., Cai, P., Metcalf, J., Matias, J. N. arXiv, 2024. arXiv:2406.01399
  24. PhantomLint: Principled Detection of Hidden LLM Prompts in Structured Documents. Murray, T. arXiv, 2025. arXiv:2508.17884
  25. When Names Change Verdicts: Intervention Consistency Reveals Systematic Bias in LLM Decision-Making. Basu, A., Chakraborty, P. arXiv, 2026. arXiv:2603.18530
  26. Prosa: Rubric-Based Evaluation of LLMs on Real User Chats in Brazilian Portuguese. Malaquias Junior, R., Bonás, G. K., Almeida, T. S., Abonizio, H. et al. arXiv, 2026. arXiv:2605.01630
  27. Judging the Judges: A Systematic Evaluation of Bias Mitigation Strategies in LLM-as-a-Judge Pipelines. Soumik, S. K. arXiv, 2026. arXiv:2604.23178
  28. Self-Preference Bias in Rubric-Based Evaluation of Large Language Models. Pombal, J., Rei, R., Martins, A. F. T. arXiv, 2026. arXiv:2604.06996
  29. Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models. Verga, P., Hofstätter, S., Althammer, S., Su, Y. et al. arXiv, 2024. arXiv:2404.18796
  30. Generalization error of ensemble estimators. Ueda, N., Nakano, R. Proceedings of the International Conference on Neural Networks, 1996. doi:10.1109/icnn.1996.548872
  31. Correlated Errors in Large Language Models. Kim, E., Garg, A., Peng, K., Garg, N. arXiv, 2025. arXiv:2506.07962
  32. RoPoLL: Robust Panel of LLM Judges. Acharya, A., Pan, K. W., Verkhovsky, B. arXiv, 2026. arXiv:2606.30931
  33. Defeating Nondeterminism in LLM Inference. He, H. et al., Thinking Machines Lab. Blog, 2025. thinkingmachines.ai
  34. The Necessity of Setting Temperature in LLM-as-a-Judge. Li, L., Sleem, L., Xu, Y., Song, Y., Jia, A., Francois, J., State, R. arXiv, 2026. arXiv:2603.28304
  35. The Comparative Trap: Pairwise Comparisons Amplifies Biased Preferences of LLM Evaluators. Jeong, H., Park, C., Hong, J., Lee, H. et al. arXiv, 2024. arXiv:2406.12319
  36. Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias. Xu, Z., Li, S., Liu, H., Wang, X. et al. arXiv, 2026. arXiv:2607.11871
  37. BACON: Budgeted Human Calibration for Modeling and Evaluation with Multiple AI Judges. Shi, L., Zhang, A., Lyu, R., Hu, Z. et al. arXiv, 2026. arXiv:2607.16239
  38. Human-anchored longitudinal comparison of generative AI with a bias-calibrated LLM-as-judge. Wiese, T. L. PLoS ONE, 2026. doi:10.1371/journal.pone.0339920
  39. Applied and Filtered: An End-to-End Algorithmic Fairness Audit of A Public Employment Agency. Galdon-Clavell, G. arXiv, 2026. arXiv:2608.13022
  40. Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreement. Jung, J., Brahman, F., Choi, Y. arXiv, 2024. arXiv:2407.18370
  41. Enforcement of Local Law 144: Automated Employment Decision Tools. Office of the New York State Comptroller. Audit report, 2 December 2025. Accessed 16 September 2026. osc.ny.gov
  42. Annex III: High-Risk AI Systems Referred to in Article 6(2), Regulation (EU) 2024/1689. European Union; text as published by the European Commission's AI Act Service Desk. Regulation, 2024. Accessed 16 September 2026. ai-act-service-desk.ec.europa.eu
  43. Colorado Enacts Law Regulating High-Risk Artificial Intelligence Systems. American Bar Association, Business Law Today. July 2024. Accessed 15 September 2026. americanbar.org
  44. AI Verify: Singapore's AI Governance Testing Initiative Explained. Future of Privacy Forum. 2023. Accessed 16 September 2026. fpf.org
  45. Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. National Institute of Standards and Technology. January 2023. Accessed 16 September 2026. nvlpubs.nist.gov
  46. ISO/IEC 42001:2023, Information technology, Artificial intelligence, Management system. International Organization for Standardization. December 2023. Accessed 16 September 2026. iso.org
  47. HireVue leads industry in fair and ethical hiring practice, engaging external auditor DCI Consulting Group for external bias audit of algorithms. HireVue, Inc. Press release, 2023. Accessed 15 September 2026. hirevue.com
  48. Mapping the use of large language models in hiring decisions: a scoping review. Tripathi, A., Tripathi, A., Dařena, F., Mishra, P. K. Frontiers in Artificial Intelligence, 2026. doi:10.3389/frai.2026.1798519
  49. AI generates covertly racist decisions about people based on their dialect. Hofmann, V., Kalluri, P., Jurafsky, D., King, S. Nature, 2024. doi:10.1038/s41586-024-07856-5
  50. Fair outputs, Biased Internals: Causal Potency and Asymmetry of Latent Bias in LLMs for High-Stakes Decisions. Tripathy, J., Buckmann, M. arXiv, 2026. arXiv:2605.15217
  51. Reasoning Models Don't Always Say What They Think. Chen, Y., Benton, J., Radhakrishnan, A., Uesato, J. et al. arXiv, 2025. arXiv:2505.05410
  52. Do Repetitions Matter? Strengthening Reliability in LLM Evaluations. Alvarado Gonzalez, M. A., Bruno Hernandez, M., Peñaloza Perez, M. A., Lopez Orozco, B. arXiv, 2025. arXiv:2509.24086
  53. Non-Determinism of "Deterministic" LLM Settings. Atil, B., Aykent, S., Chittams, A., Fu, L. arXiv, 2024. arXiv:2408.04667
  54. Fragile preferences: A deep dive into order effects in large language models. Yin, H., Vardi, S., Choudhary, V. PNAS Nexus, 2026. doi:10.1093/pnasnexus/pgag246
  55. A Two-Phase Stability Study of LLM Judges and Bar Council Examiners on Thai Bar-Exam Free-Form Essays. Akarajaradwong, P., Lertprasertphakorn, W., Chaksangchaichot, C., Nutanong, S. arXiv, 2026. arXiv:2605.25652
  56. Artificial Intelligence-Assisted Narratives: Analysis of Surgical Residency Personal Statements. Subillaga, O., Pérez Coulter, A., Tashjian, D., Seymour, N. et al. Journal of Surgical Education, 2025. doi:10.1016/j.jsurg.2025.103566
  57. AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights. Xu, J., Li, G., Jiang, J. Y. arXiv, 2025. arXiv:2509.00462
  58. Regulation (EU) 2026/1744 amending Regulation (EU) 2024/1689 (the "Digital Omnibus" on artificial intelligence), recital 40 and Article 1. European Parliament and Council. Official Journal, 24 July 2026; in force 27 July 2026. Accessed 16 September 2026. eur-lex.europa.eu
  59. Automated Employment Decision Tools (AEDT): Frequently Asked Questions. New York City Department of Consumer and Worker Protection. Guidance document. Accessed 16 September 2026. nyc.gov
  60. SB26-189: Consumer Protections for Artificial Intelligence, bill summary and status. Colorado General Assembly. Signed 14 May 2026. Accessed 16 September 2026. leg.colorado.gov
  61. What is the EEOC's role in AI? U.S. Equal Employment Opportunity Commission. April 2024. Accessed 15 September 2026. eeoc.gov
  62. iTutorGroup to Pay $365,000 to Settle EEOC Discriminatory Hiring Suit. U.S. Equal Employment Opportunity Commission. Press release, 2023. Accessed 15 September 2026. eeoc.gov
  63. The Changing Landscape of AI: Federal Guidance for Employers Reverses Course with New Administration. K&L Gates. 31 January 2025. Accessed 16 September 2026; the EEOC page it describes now returns "page not found". klgates.com
  64. Federal Court Allows Collective Action Lawsuit Over Alleged AI Hiring Bias (Mobley v. Workday). Holland & Knight LLP. May 2025. Accessed 16 September 2026. hklaw.com
  65. California Federal Court Grants in Part and Denies in Part Workday's Motion to Dismiss in Mobley v. Workday. Duane Morris LLP, Class Action Defense Blog. 24 June 2026. Accessed 16 September 2026. duanemorris.com
  66. HB3773, 103rd General Assembly, bill status (Public Act 103-0804). Illinois General Assembly. Effective 1 January 2026. Accessed 16 September 2026. ilga.gov
  67. Tripartite Guidelines on Fair Employment Practices. Tripartite Alliance for Fair and Progressive Employment Practices, Singapore. Accessed 15 September 2026. tal.sg
  68. Use of Artificial Intelligence in Recruitment Tools and Singapore's Workplace Fairness Act. K&L Gates. 23 December 2025. Accessed 16 September 2026. klgates.com
  69. Artificial Intelligence (AI): Guidance for Judicial Office Holders. Courts and Tribunals Judiciary, United Kingdom. Issued December 2023; version of 31 October 2025. Accessed 16 September 2026. judiciary.uk
  70. Texas will use computers to grade written answers on this year's STAAR tests. The Texas Tribune. 9 April 2024. Accessed 16 September 2026. texastribune.org
  71. Investigating first-language bias in LLM-based automated essay scoring: A cross-prompt evaluation of an open-weight AI-model on TOEFL essays. Gayed, J. M. arXiv, 2026. arXiv:2607.14605
  72. Implicit Grading Bias in Large Language Models: How Writing Style Affects Automated Assessment Across Math, Programming, and Essay Tasks. Jadhav, R., Danve, J., Shaw, S. arXiv, 2026. arXiv:2603.18765
  73. Institutional Prestige as Geographic Bias in Large Language Models: Evidence from Three Factorial Experiments with Bootstrap Confidence Intervals. Leyva-Vázquez, M., Smarandache, F. arXiv, 2026. arXiv:2608.18107
  74. Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models. Zhu, Y., Yang, Z. arXiv, 2026. arXiv:2606.23030
  75. Consumer Financial Protection Circular 2023-03: Adverse action notification requirements and the proper use of the CFPB's sample forms provided in Regulation B. Consumer Financial Protection Bureau. 19 September 2023. Accessed 16 September 2026. consumerfinance.gov
  76. Cigna sued over algorithm allegedly used to deny coverage to hundreds of thousands of patients. STAT News. 24 July 2023. Accessed 16 September 2026. statnews.com
  77. CMS Launches New Model to Target Wasteful, Inappropriate Services in Original Medicare (the WISeR model). Centers for Medicare and Medicaid Services. Press release, 27 June 2025. Accessed 16 September 2026. cms.gov
  78. New Records Reveal Problems with Medicare's AI Prior Authorization Experiment. Electronic Frontier Foundation. September 2026. Accessed 16 September 2026. eff.org
  79. Tax office fined 2.75 million euros for privacy violations in benefits scandal. NL Times. 8 December 2021. Accessed 16 September 2026. nltimes.nl
  80. Report of the Royal Commission into the Robodebt Scheme. Royal Commission into the Robodebt Scheme, Commonwealth of Australia. 7 July 2023. Accessed 16 September 2026. robodebt.royalcommission.gov.au
  81. Robodebt condemned by Federal Court judge as "shameful chapter" as settlement approved. ABC News (Australia). 11 June 2021. Accessed 16 September 2026. abc.net.au
  82. Home Office says it will abandon its racist visa algorithm after we sued them. Foxglove. 4 August 2020. Accessed 16 September 2026. foxglove.org.uk
  83. Colombian judge uses ChatGPT in ruling on child's medical rights case. CBS News. February 2023. Accessed 16 September 2026. cbsnews.com
  84. Grassley Releases Judges' Responses Owning Up To AI Use, Calls For Continued Oversight And Regulation. Office of Senator Chuck Grassley. October 2025. Accessed 16 September 2026. grassley.senate.gov
  85. Snell v. United Specialty Insurance Co., No. 22-12581 (11th Cir. 28 May 2024), concurrence of Judge Newsom. United States Court of Appeals for the Eleventh Circuit. Accessed 16 September 2026. law.justia.com
  86. Opinions on Regulating and Strengthening the Applications of Artificial Intelligence in the Judicial Fields. Supreme People's Court of the People's Republic of China. 8 December 2022. Accessed 16 September 2026. court.gov.cn
  87. "Positive review only": Researchers hide AI prompts in papers. Nikkei Asia. 1 July 2025. Accessed 16 September 2026. asia.nikkei.com
  88. Leveraging LLM Feedback to Enhance Review Quality. ICLR 2025 Programme Chairs. ICLR blog, 15 April 2025. Accessed 16 September 2026. blog.iclr.cc
  89. AAAI Launches AI-Powered Peer Review Assessment System. Association for the Advancement of Artificial Intelligence. Accessed 16 September 2026. aaai.org
  90. Pangram predicts 21% of ICLR reviews are AI-generated. Pangram Labs. 18 November 2025. Accessed 16 September 2026. pangram.com
  91. AI-Assisted Reviewing Experiment. NeurIPS 2026. Accessed 16 September 2026. neurips.cc
  92. Using GPT-4 for content moderation. OpenAI. 15 August 2023. Accessed 16 September 2026. openai.com
  93. More Speech and Fewer Mistakes. Kaplan, J., Meta. 7 January 2025. Accessed 16 September 2026. about.fb.com
  94. Federal workers, agencies push back on Elon Musk's email ultimatum. NBC News. 24 February 2025. Accessed 16 September 2026. nbcnews.com