How AI Content Detectors Work, and Why They Flag Human Writing as AI

The paper that came back 90 percent machine

In the spring of 2023, Taylor Hahn was grading papers for a communications course at Johns Hopkins University. He ran one submission through Turnitin, the plagiarism service used by more than 16,000 institutions worldwide, which had switched on an AI writing detector that April. The tool reported that over 90 percent of the paper had been written by a machine.

Hahn did the careful thing. He set up a Zoom call and asked the student to walk him through her notes and the material she had worked from.

Then he kept grading. Over the rest of that semester he noticed something the score itself could never have surfaced. The flags kept landing on international students.

That pattern is where this article ends up, and it only makes sense once you can see the machine producing it. So we will build the machine first and come back to the classroom in the second half.

The claim being defended here is a narrow one. AI detectors do not misfire in spite of how they work. They misfire because of how they work. A false accusation is the system running correctly.

What a detection score actually means

Every detector hands you a number, and almost every problem downstream begins with people reading that number wrong.

A result of "92% AI" does not mean 92 percent of the document was machine-written. It means the model holds 92 percent confidence in a guess. The tool never watched the document being written. It has no access to the writer, no draft history, no record of revisions and no evidence in any sense a disciplinary hearing would recognise. What it has is a set of statistics about word patterns and a threshold that somebody chose.

Turnitin states this more plainly than most of its customers repeat it. The company tells educators to treat highlighted sentences as areas of interest, close to where AI writing may sit, and to use the result to open a conversation rather than to reach a conclusion.

Keep that distinction close. Everything in the sections below follows from people treating a probability as a verdict.

A detector is not a plagiarism checker

The two tools get grouped together because they often ship in the same product, but they answer different questions and carry different levels of certainty.

A plagiarism hit points at something. An AI score points at nothing, which is what makes it so difficult to argue with.

The four signals every detector reads

Detection tools differ in branding far more than in method. Nearly all of them read some mix of the four signals below.

Perplexity: how surprising is the next word?

Perplexity measures predictability. Feed text to a language model one word at a time and ask, at each step, how surprised the model is by what actually comes next. Little surprise means low perplexity.

Large language models are trained to produce likely text, so their output clusters at the low end of that range. Detectors convert the observation into a rule: predictable writing looks like machine writing.

Read the rule once more, because the entire failure mode of this technology is folded inside it. The tool is not detecting machines. It is detecting predictability and then assuming machines.

Burstiness: does the rhythm vary?

Burstiness measures how much sentence length and structure vary across a passage. Human prose tends to lurch. A four-word sentence lands after a thirty-word one. Clauses interrupt. Fragments appear where a grammar teacher would object.

Machine prose holds a steadier beat, with most sentences clustering near the same length. Count the words per sentence across a paragraph and the difference becomes visible.

Classifiers: supervised pattern matching

A classifier is a model trained on labelled examples of human and machine text until it can separate the two. Early tools used logistic regression, decision trees, support vector machines or random forests. Newer ones fine-tune transformer models on the same task.

The output is a confidence score whose reliability moves with text length, subject matter, genre and how heavily the passage has been edited. Short passages give a classifier almost nothing to work with, which is why Turnitin raised its minimum document length from 150 words to 300 and reported that accuracy improves with more text.

Embeddings and stylometry

Embeddings represent words as vectors so a model can compare meaning rather than spelling. Stylometry sits alongside it and reads habits a writer never consciously chooses: how often function words appear, where clauses tend to break, which punctuation marks recur and how paragraphs are proportioned.

Neither signal knows anything about authorship. Both infer it from surface texture, which is a reasonable thing to do and a fragile thing to punish someone over.

Watermarking was supposed to solve this

If detection by inference is fragile, detection by design should be sturdier. That is the promise behind watermarking.

Google's SynthID nudges the probability distribution over tokens during generation, planting a statistical signal in the output that a matching detector can look for afterwards. Nothing is guessed at. The signal was placed there deliberately.

Four limits stop it closing the case:

•     Detection requires access to the configuration used at generation time, which puts it beyond the reach of most third parties.

•     The signal survives light editing but degrades sharply under thorough rewriting or translation into another language.

•     Watermarking works least well on factual answers, where there is less room to shift word choice without harming accuracy.

•     It only covers models whose makers opted in. Everything else leaves no mark at all.

Google's own documentation describes watermark detection as probabilistic and warns that confidence scores fall considerably once text has been rewritten.

So the deliberate signal covers a slice of the problem and inference covers the messier remainder. The next section is about what happens in that remainder.

Where the machine starts misfiring

Return to Hahn's classroom.

Think about what perplexity rewards: unusual word choices, syntax that wanders, phrasing a model would not have picked. Now think about who writes without those qualities. Someone working in a second language, leaning on the constructions they were drilled in, reaching for the safe word instead of the surprising one.

Their writing is predictable. The rule from the previous section fires. Nothing malfunctioned.

The predictability penalty

In 2023, a Stanford team led by Weixin Liang tested seven GPT detectors against 91 TOEFL essays, every one of them written by a human. They ran the same detectors over essays by US eighth-graders who were native English speakers.

The detectors handled the native-speaker essays close to flawlessly. On the TOEFL essays, the average false positive rate reached 61.3 percent. At least one detector flagged 97.8 percent of them. All seven agreed, wrongly, on 19.8 percent.

The finding is contested, and honesty requires saying so. Turnitin criticised the study for using a small sample of short texts and reported no comparable bias in its own testing on longer documents. Pangram, a newer detector, holds the TOEFL set out of its training data and reports a 0 percent false positive rate on it, while noting that GPTZero's updated model still scored 7.7 percent on the same benchmark.

The mechanism does not rest on any single study, though. Low perplexity is what these tools hunt for, and second-language prose produces it reliably.

The same logic reaches further than international students. University guidance now flags neurodivergent writers as elevated-risk, since repeated phrasing and consistent structure read to a detector as machine uniformity. Legal writers, technical documentarians, government communicators and anyone trained to write plainly sit in the same zone. Clear writing scores as suspicious writing.

The base rate problem

Turnitin's advertised document-level false positive rate was under 1 percent. That sounds like a solved problem until you multiply it out.

Vanderbilt University ran exactly that multiplication in August 2023. It had submitted roughly 75,000 papers to Turnitin during 2022. At a 1 percent error rate, around 750 of those papers would have been wrongly labelled as containing AI writing.

Vanderbilt disabled the detector. Its published reasoning named four problems: the volume of likely false positives, the absence of any explanation of how the tool reaches a verdict, the documented bias against non-native English writers, and the fact that the feature had been switched on with under 24 hours of notice and no way to turn it off.

Earlier that same year, Turnitin's chief product officer had already complicated the picture. The sub-1 percent figure applies to documents containing 20 percent or more AI writing. The sentence-level rate, meaning the chance that any single highlighted sentence was in fact written by a person, sits at around 4 percent. False positives cluster in the first and last few sentences of a document, which is to say in the introduction and the conclusion.

Detectors do not travel

A detector's score on the benchmark it was tuned against tells you little about how it behaves on your inbox.

A 2026 analysis tested this directly, training linguistic-feature classifiers and evaluating them across two benchmark corpora. On PAN-CLEF 2025 the model reached an F1 of 0.9734. Moved to the COLING 2025 corpus, the same approach dropped to 0.8025.

Using SHAP to inspect which features drove the decisions, the researchers found that the most influential features differed markedly between the two datasets. The classifiers had learned the stylistic fingerprints of particular corpora rather than a stable signature of machine authorship.

This is why the problem compounds rather than settles. Every model release shifts the distribution again, and detectors are always trained on the generation that came before. Add human editing on top of machine drafting, which is now how most AI-assisted writing is actually produced, and the clean two-way split the classifier was trained on stops existing.

The asymmetry nobody advertises

Paraphrasing defeats most detectors. So does inserting a personal anecdote, varying sentence length on purpose, dropping in a typo or two, or running the output through a second tool built for that job.

The result is an inversion worth sitting with. A student who writes cleanly in their own voice and submits their own work gets flagged. A student who generates an essay and spends ten minutes roughening it up sails through.

These tools are hardest on the honest.

What the vendors' own numbers admit

Detection companies publish more damaging information than their critics usually notice.

OpenAI shipped an AI text classifier in January 2023 with an unusually frank caveat attached. On its own challenge set of English texts, the tool correctly identified 26 percent of AI-written text while labelling human-written text as AI 9 percent of the time. It was withdrawn on 20 July 2023 for low accuracy, and the company said it was researching provenance techniques instead.

Set the rest of the field's public claims side by side and a pattern appears in the right-hand column.

Every strong number in that table is a benchmark number. Benchmarks are in-distribution by construction, and the previous section showed what happens when a detector leaves the distribution it was built for.

The institutions are backing away

Vanderbilt was first to publish the arithmetic. Others reached the same conclusion and said so.

MIT Sloan's teaching guidance states plainly that AI detectors do not work reliably enough for serious use. UC Berkeley declined to enable detection, citing student privacy and the risk of handing coursework to a third party. The University of Waterloo cited peer-reviewed studies by name alongside an internal test in which human-written text came back scored as fully AI. Curtin University switched detection off in January 2026 and framed the decision as education rather than surveillance.

The harm keeps surfacing in the meantime. ABC News reported that Australian Catholic University used AI detection to accuse roughly 6,000 students of academic misconduct in 2024, with many cleared only after long delays. One nursing student spent six months with "results withheld" showing on her transcript while her case was reviewed.

Using a detector without wrecking someone's semester

These tools still have a use. Their proper place is at the start of an enquiry, and everything depends on what you do with the number next.

If you assess student work: treat a score as a reason to ask a question, never as an answer. Ask the student to talk through the argument and the choices they made along the way. Someone who wrote the paper can do that in five minutes. Someone who did not will struggle, and you will have learned something a percentage could never tell you. Remember that an accusation costs a student real time and standing even when they are eventually cleared.

If you commission or edit content: build your standard around process instead of scores. A brief, an outline, a draft with visible revisions and a writer who can defend the piece tell you more than any detection percentage. Writing a scanner threshold into a freelance contract mostly guarantees that your best plain-English writers will fail it.

If you write and get flagged: keep the evidence before you need it. Version history in Google Docs or Word, dated drafts, research notes and any tool that records how a document was assembled are the strongest defence available, because they document the process the detector never saw. Start now, while nobody has accused you of anything.

That last point explains where this market is heading. Grammarly Authorship logs what was typed by hand and what was pasted in from somewhere else. The C2PA standard attaches signed provenance metadata at the moment content is created, and camera manufacturers have started shipping support for it. Both approaches skip the question of what finished text looks like and record what actually happened while it was being made.

The industry built inference-based detection because provenance did not exist yet. The pivot toward provenance is a quiet admission of what the detection numbers have been saying since OpenAI pulled its own classifier off the shelf in July 2023.