Detecting AI in a student’s work: why reliability is not the question

A text comes in that does not sound like the student. The vocabulary is too assured, the sentences too even, and the teacher has a feeling, hard to put into words, that someone else wrote it. The reflex is to look for a tool that will settle the matter. It is that reflex I want to question, not because detectors are bad, since they have improved a great deal, but because of the kind of answer they give.

1.What the tools showed between 2023 and 2024

OpenAI released its own classifier on 31 January 2023 and withdrew it on 20 July of the same year, stating the reason on the page itself: accuracy was too low. The figures the company published are worth recalling. The tool correctly identified 26% of AI-written texts, and wrongly flagged 9% of human-written ones (OpenAI, 2023).

That same year, an international team tested fourteen detection tools on fifty-four purpose-built cases (Weber-Wulff et al., 2023). None reached 80% accuracy. Using the binary approach, mean accuracy fell to 30% on AI text edited by hand and to 15% on AI text run through an automatic paraphraser. The mean false accusation rate was 2.4% on human texts, rose to 11.1% on machine-translated ones, and reached 50% for one tool in particular. The authors are explicit: these systems should not be used in academic settings.

A piece published the same year in Patterns, which the journal presents as an opinion rather than a research article, added an inequality. Seven detectors applied to 91 essays written by non-native English speakers flagged on average about 61% of them as AI-generated, against roughly 5% for essays by American school pupils (Liang et al., 2023). Asking a model to rewrite those same essays in more idiomatic English brought the rate down to about 12%.

A year later, another team showed that very simple manipulations are enough to collapse detection (Perkins et al., 2024). Mean accuracy fell from 39.5% on unmodified AI content to 22.1% after manipulation. The most effective techniques were, in order, adding spelling errors, increasing the unevenness of the text, then paraphrasing.

2.But the picture has changed, and that needs saying

Stopping there would be convenient and dishonest. A 2026 study tested four detectors on 160 documents of known origin (Van Vlasselaer et al., 2026). All four classified 100% of the human texts correctly, and the authors note that false positives are now rare, unlike in earlier work. On hybrid texts, or texts reworked to look human, the best-performing tool reached 92.5% accuracy.

In other words, the argument that detectors get it wrong too often is going out of date. Citing only the 2023 work would be choosing one’s sources. And yet the same authors conclude that these tools should not serve as the sole evidence in a high-stakes decision. That is the conclusion that interests me, because it does not depend on how well the tool performs.

3.A good detector does not solve the problem

A detector produces a probability about a text. A disciplinary decision is about a person and an act. Between the two, something is missing that the tool cannot supply, however accurate it is.

The reasoning is simple, and it is reasoning, not a study result. Suppose an excellent tool, one that very rarely misjudges a human text. Applied to a set of papers of which only a small share were produced by an AI, it will still produce a few wrong flags. The problem is not how many there are. It is that nothing in the tool’s output separates the correct flag from the mistaken one. That is precisely what a probability means.

From the student’s side it is worse. Asked to prove they wrote their own text, an honest student has nothing to produce. There is no proof of absence. The burden of proof ends up on the wrong side, on the person least equipped to carry it.

The Romandie recommendations say the same thing in a sentence: detection tools, themselves based on probabilistic methods, are not in a position to assess reliably whether generative AI was used (CIIP, 2025).

4.Move the question rather than settle it

If detection cannot ground a decision, tasks have to be designed so that the question arises differently. Three directions seem sound to me.

• Name the rules beforehand, not afterwards. The CIIP proposes a teaching contract between teacher and students, setting out permitted uses, the transparency expected, and what counts as cheating. A rule stated in advance needs no detector in order to be applied.
• Assess what can be checked. Not “was this text written by an AI?” but “does this student command what they handed in?”. The HEP Vaud directive on certifying assessments provides a mechanism of this kind: the panel may call the student for an oral on a short extract of their own work, chosen by the panel, and will in principle record a fail if the student cannot demonstrate the coherence and accuracy of that extract (HEP Vaud, 2025). The question becomes checkable, and the student can answer it.
• Shift the weight towards the process. This is the CIIP’s explicit recommendation for assessment: refocus on how the work was done rather than on the finished product alone, and accept that in homework, banning the tool is impossible.

None of these is comfortable. They take more time than a button that returns a percentage. But they turn on questions a student can answer, which the first one does not.

Let’s talk

How does your school handle the text that does not sound like the student? I would be glad to discuss it. My contact details: pablocorreaprieto.ch

Text written with the help of AI, reviewed and approved by the author.