How AI Detection Actually Works: Signals, Scores and Limits

An AI detector never sees the writing happen. It receives a finished string of text and estimates how well that string matches the way language models tend to produce text. Everything a detector can and cannot do follows from that single constraint.

This article explains the mechanisms that are actually used, what each one measures, and why the ceiling on accuracy is structural rather than a matter of engineering effort.

The core idea: predictability

A language model generates text by repeatedly picking a likely next token. Depending on the decoding settings, it tends to pick tokens that sit high in its own probability ranking. The result is prose that is, statistically, less surprising than human writing of the same kind.

Human writing is more uneven. People change register mid-paragraph, reach for an unexpected word, leave a sentence short, repeat themselves, and make choices a model would have scored as unlikely. Detectors are built on that gap.

The consequence is immediate: anything that flattens a human’s unevenness — a taught essay template, a translation pass, careful second-language writing, heavy editing — moves that text toward the same region a detector treats as machine-like. That is not a bug in one vendor’s implementation. It is what the measurement is.

The four families of signal

Every serious detector combines some subset of these. They fail in different ways, which is the main argument for using more than one.

1. Token probability and rank

The oldest approach: score each token by how probable it was under a reference language model. GLTR made this visible by colouring tokens by rank, and its study reported that giving readers this visualisation improved their own detection rate from 54% to 72%. It is a tool for showing evidence, not a classifier.

Raw perplexity alone is a fragile signal. It depends on which reference model you use, and it shifts with language, domain and the author’s linguistic range — which is exactly how it becomes a proxy for fluency instead of authorship.

2. Curvature and contrast methods

DetectGPT starts from a sharper observation: model outputs tend to sit in regions of negative curvature in the model’s log-probability function. It perturbs the passage and compares scores, so it needs no training data — but it needs the scoring model, a perturbation model and several inference passes per document.

Fast-DetectGPT replaces perturbation with conditional sampling and reports a large speedup in its own protocol. Binoculars contrasts the perplexity of one model against the cross-entropy given by a second, closely related model. Both are zero-shot; both need access to model logits, which means real GPU inference rather than a browser or an edge function.

3. Trained classifiers

A supervised classifier learns the difference from labelled examples. Ghostbuster combines features from several weaker language models before a final classifier, and does not require probabilities from the target generator. This family performs best when test data resembles training data, and generalises poorly to unseen generators, domains and attacks.

That caveat matters more than the headline scores. A classifier trained before a model release has never seen how the new model writes.

4. Stylometric and structural features

Sentence-length variation, connector sequences, syntactic parallelism, repeated openings, mechanically regular lists, formulaic vocabulary. These are weak on their own and easy to game, but they are the only family that can explain a result in words a reader can check.

No single word, punctuation mark or em dash is evidence of anything. Claims that one character proves AI authorship are folklore.

From signals to a score

The individual signals are not probabilities a user can read. Turning them into a score requires two more steps that are usually invisible and usually where the quality difference sits.

Aggregation. A meta-model combines module outputs. Nothing should produce a verdict alone, and a generative model asked “does this look like AI?” is a weak feature, not a source of truth.

Calibration. The combined score has to be mapped to a reported number separately per language and per length band, because a raw score means different things on a 200-word CV and a 3,000-word thesis. Thresholds should be chosen at a fixed false-positive rate — which is what the RAID benchmark protocol does — rather than at maximum accuracy, because maximum accuracy quietly buys correctness on generated text with errors against real people.

A well-built detector therefore has three outputs, not two: likely human, uncertain, and consistent with AI generation. A tool that never says “uncertain” is not more confident; it is hiding the middle.

Why sentence-level attribution is a separate problem

A document score cannot simply be copied onto each sentence. Mixed documents — a human draft with generated paragraphs, or generated text a person rewrote — defeat document-level methods, and SeqXGPT builds a dedicated sentence-level model precisely because the document approach does not decompose.

This is why the honest label for a highlighted span is “passage to review”, not “written by AI”. The highlight tells you where the score came from. It does not attribute a sentence to a machine. That framing is what reading a detection score depends on.

Watermarking: real, and rarely available

Watermarking is the one method that is not statistical guesswork. The generator biases its own token selection according to a key, and a detector holding that key can test for the pattern. Google DeepMind’s SynthID-Text is a production implementation of the same idea.

The catch is that it only works when the generator applied the watermark and you have the detector for it. There is no universal watermark, no way to check a document for “any AI watermark”, and robustness degrades under paraphrase — the follow-up reliability study reports needing roughly 800 tokens at a 10⁻⁵ false-positive rate after heavy human paraphrasing.

Without a protocol and a key, the correct answer from a provenance module is “not verifiable”. Anything else is a guess wearing a certificate.

Why there is a ceiling

Two results frame the whole field. The first is empirical: Sadasivan and colleagues showed that recursive paraphrasing breaks a broad range of detectors, including watermarked and retrieval-based ones. The second is theoretical: as the distribution of generated text approaches the distribution of human text, the best achievable performance of any text-only detector degrades toward chance.

Better models make detection structurally harder, not easier. That is the opposite of how most software problems behave, and it is why “detection will improve until it is reliable” is not a safe assumption to build a policy on.

Broad evaluations agree. RAID spans millions of generations across multiple models, domains, decoding strategies and adversarial attacks, and reports failures under changes in any of them.

What this means for using a detector

The defensible claim is narrow and worth stating exactly: a detector measures patterns consistent with AI generation, and shows why they were flagged. It does not prove authorship, identify which model wrote a text, or produce a trustworthy percentage of AI-written words.

Used that way — as triage that tells a person where to look — detection is genuinely useful. Used as a verdict, it converts a known error rate into decisions about people. See how accurate AI detectors really are for the numbers, and why AI detectors flag human writing for the failure modes. Then open the AI detector and read the passages, not only the percentage.

How AI detection actually works