AI detectors do not fail at random. They fail more often on writers who learned English as a second language, and the reason is structural rather than accidental. If you run detection on essays, applications or submissions from an international cohort, this is the bias you have to plan around.
This is uncomfortable for a company that sells detection. We publish it because a reviewer who does not know this will make worse decisions with our tool than with no tool at all.
What the research actually shows
Liang and colleagues evaluated commercial GPT detectors on two human-written corpora: essays by US eighth-grade students, and TOEFL essays written by non-native English speakers. The detectors classified the eighth-grade essays with near-perfect accuracy. More than half of the non-native TOEFL essays were misclassified as AI-generated.
Every text in both corpora was written by a person. The only variable that moved was who wrote it. The study was published in Patterns in 2023 and remains the most cited evidence on this point.
The detectors were not measuring whether a machine wrote the text. They were measuring how the text was written.
Why it happens
Most text detectors reduce to one underlying question: how predictable is this sequence of words? Generated text tends to sit in the high-probability, low-variance region of a language model’s distribution. Detectors learn to treat that region as suspicious.
Careful second-language writing lives in the same region, for entirely different reasons. A writer with a smaller active vocabulary reaches for the common word. A writer taught a formal essay template uses regular connectors and even sentence lengths. A writer who checks each sentence before moving on produces prose with less variation than a fluent native speaker writing quickly.
The Liang study tested this directly: enriching word choice in the non-native samples reduced misclassification, while simplifying the native samples increased it. The signal being picked up is linguistic range, not authorship.
The same mechanism explains the other frequent false positives — translated documents, technical prose, and heavily edited drafts. We cover the general case in why AI detectors flag human writing.
Who this affects in practice
- International students. The cohort already under the most scrutiny in academic integrity procedures, and the least equipped to contest a technical claim in a second language.
- Job applicants. Cover letters and CVs are formulaic by convention and short by convention — two independent false-positive risks stacked on top of the language one.
- Researchers publishing in English. Academic English written by non-native speakers, often with translation assistance, combines every factor at once.
- Support and operations teams. Templated professional English from distributed teams reads as uniform to a detector.
Vanderbilt University cited this bias among its reasons for disabling Turnitin’s AI detector in August 2023, alongside a lack of transparency about how the detection works. That decision is a reasonable reading of the evidence, not an overreaction.
If you review other people’s writing
- Do not run detection as a screen. Scoring every submission and escalating the top of the list concentrates your errors on second-language writers by construction.
- Set a length floor. Short samples produce noise. A 400-word cover letter is not a document you can responsibly score.
- Read the flagged sentences. Ask whether the highlighted spans are formulaic openings and transitions — the fingerprint of a taught template — or the substantive argument.
- Ask process questions, not language questions. Drafts, sources, notes and an explanation of the argument are evidence a score cannot produce. “This doesn’t sound like you” is not.
- Never let a score be the sole basis for a penalty. This is the rule that makes the rest of the process survivable when the detector is wrong.
For the classroom version of this process, see AI detection in schools and the guide for teachers. For hiring, see recruiters.
If your own writing was flagged
Ask for the specific passages rather than arguing with the percentage. Explain how those sentences were written: the source you were summarising, the outline you worked from, the phrasing you were taught to use. Show version history, drafts or notes if you have them.
Do not run the text through a humanizer to lower the score. It changes writing you are being asked to account for, and it looks like concealment if the change is noticed. The full process is in wrongly accused of using AI: what to do.
What this means for IA Checker
We cannot claim to have solved this. No text-only detector has. What we can do is refuse to hide it: report sentence-level evidence you can argue with, keep an uncertain state instead of forcing a binary verdict, decline strong conclusions on short samples, and audit false positives on non-native writing separately rather than inside an aggregate.
Language coverage in the public evaluation literature is uneven — MULTITuDE, one of the larger multilingual detection corpora, spans eleven languages and does not include French. Any detector claiming uniform performance across languages is claiming something nobody has measured. The AI detector reports indicators, and a person stays in the decision.
Sources
- Liang, Yuksekgonul, Mao, Wu and Zou — GPT detectors are biased against non-native English writers (preprint)
- The same study in Patterns, Cell Press (2023)
- Vanderbilt University — Guidance on AI detection and why we’re disabling Turnitin’s AI detector (August 2023)
- Macko et al. — MULTITuDE: large-scale multilingual machine-generated text detection benchmark