← library1234567 words font

Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift

Kevin Ren, Manish Raghavan, Nikhil Garg · NeurIPS 2026 (arXiv 2606.25152, v2) 2026

about 10 minthe paper (PDF)reviewnotes written 2026-11-05

AI text detectors are trained once and then released, but the text they see keeps changing. Can a detector that keeps learning from the unlabeled texts it receives hold up where a fixed one fails?

key idea what the paper found what is lost, or a limit

chapter 1

The question in one picture

Pangram is a commercial AI text detector. On abstracts that a large language model (LLM) rewrote in the ordinary way, it caught 96% of the AI text. §4.1 The authors then had an agent search for a rewrite instruction that Pangram misses. On text written with that instruction, it caught 27%. §4.1

Figure 1 shows why. Pangram is a supervised detector: it learns one line between human text and AI text from labeled examples. Then it is released and stays fixed. Figure 1 The text it sees keeps changing after release. People rewrite AI text to dodge it. New LLMs come out. Human writing itself drifts. §1

Figure 1 from the paper: a fixed detector and an adapting detector Figure 1 open full size

Figure 1 from the paper: a fixed detector and an adapting detector

Top row: a detector is trained once on labeled human and AI text, then released.

After release, the text changes: rewriting tools that dodge detectors, new LLMs, and human writing that drifts.

The released detector stays fixed. Shifted AI text (orange rings) lands on the human side of its line.

Bottom row: the detector adds the texts it receives, with no labels, and redraws its line. The shifted AI text is now on the AI side.

The paper asks whether a detector can keep up by learning from the texts people send it, without labels. §1

The answer, in short

In the paper's tests, fixed detectors failed under all three kinds of change, while detectors that adapted at test time mostly held up. §4 Against the dodging text, the adapting detector caught 93.3%. Abstract The authors call it a promising framework, not a cure. §6

check yourself

Pangram caught 96% of ordinary AI rewrites and 27% of the dodging ones. What changed between the two tests?

Why can the detector not just be retrained on the new text?

chapter 2

What you need first

How detectors are trained

Builders take human text written before 2022. For each one, they ask an LLM to rewrite it. The paper calls the rewrite a mirror. §1 The detector learns to tell human texts from mirrors. When test text matches training text, modern detectors such as Pangram are over 99% accurate. §1

The paper scores a detector two ways. AI recall is the share of AI texts it calls AI. Human recall is the share of human texts it calls human. §3

Distribution shift

A distribution shift means the test text differs from the training text. Abstract Such text is OOD, short for out-of-distribution. §4.2 One kind is a humanizer: a tool or prompt that rewrites AI text to dodge a detector. §1

The paper calls this continual distribution shift: the change keeps happening, and the new text comes with no labels. §1

Learning without labels

A detector that learns after release has a pile of unlabeled texts. §1 One tool for this is PU learning, short for positive-unlabeled learning. §2 The detector gets labeled texts of one class, the positive class. It also gets an unlabeled pile that mixes both classes. It learns to find what in the pile is not like the known class. §2

Why is this possible at all?

The paper points to test-time homogeneity. Many people use the same few models and humanizers. So the AI texts that arrive together tend to look alike. §1 One text alone may be hard to judge. A batch can show a group that does not match the known human texts. §1

The paper's example is a news report. Professors asked students to admit to banned LLM use. An unusual share of the replies said “sincerely apologize”. §1 The paper reads this as a sign that many students used an LLM to apologize. §1

check yourself

What is a mirror, and why do detector builders make them?

A detector catches 99% of AI text but calls many real abstracts AI. Which score shows the problem?

chapter 3

What the paper does

The argument: the detector moves first

A detector is released, often as a public service that anyone can send text to. §2 A user can keep editing AI text until the detector says “human”. §2 The fixed detector cannot answer until it is retrained with new labeled data, which takes time. §2

So a fixed detector always moves first, and the other side moves last. The paper argues the detector must move last, adapting to the newest text when it is tested. §2 This is TTA, short for test-time adaptation. §1

The three observations in Section 2

The paper makes the argument with three short claims. Their proofs are in Appendix C. C

  1. If human and AI text have the same distribution, no detector beats a coin flip. Accuracy is capped by how different the two are. §2
  2. Many lines may split human text from the old AI text perfectly. Some of them fail on the shifted AI text. So a detector can be perfect in training and still miss shifted text. §2
  3. With labeled human text and an unlabeled mix from test time, a PU learning goal exists whose best answers split human text from all AI text. §2

Claim 3 assumes the groups are separable: some detector of the kind being trained could split them perfectly. §2 In the paper's tests, a detector trained on matching data split them almost perfectly, even for single sentences. §2

The method: learn from the texts that arrive

The paper's main detector is PU + TTA. It is built from (TED)^n, a published method for PU learning. §3 Its classifier is DistilBERT, as in the paper's supervised detectors. Only the training goal differs. §3

How PU + TTA trains, for a humanizer test §3

labeled human textthe positive class
unlabeled texts it receiveshuman and AI, mixed
(TED)^nstep a: guess how much of the pile is human
train the classifierstep b: the pile minus its most human-like part counts as AI
label new textsheld-out texts from the same stream

The detector has labeled texts of one class only. In the humanizer test, that class is human writing.

It also has the texts people send it, with no labels. Some are human. Some are AI, maybe from a new humanizer.

(TED)^n first guesses what share of the pile looks like the known human texts.

It then trains a DistilBERT classifier. The pile, minus its most human-like share, is treated as AI. It then guesses the share again and retrains, back and forth.

The trained classifier labels held-out texts from the same stream. That is what the paper scores.

In the humanizer and new-LLM tests, human text is the positive class. B.3 In the human-drift test, the roles swap. Fresh mirrors are the known class, because new human text has no trusted labels. §4.3 A second version, PNU + TTA, also gets labeled examples of the other class. §3

The data, the other detectors, and checks

The text is arXiv abstracts, with mirrors from 11 LLMs. §3 Ordinary mirrors use the naive prompt of the method of Liang et al.. §3 Most tests score single sentences. §3

The supervised detectors are DistilBERT, Pangram 3.3.2, and Liang et al.. §3 Liang et al. estimates what share of a collection is AI text from word counts. §1 The paper also tests a zero-shot detector, a domain-adaptation method, and TTT (Test-Time Training). §3 TTT updates the model on incoming text with a fill-in-the-blank task, so it can work on a stream. §3

Pangram gets whole abstracts, because sentences are too short for its accuracy claims. §3 The split into training and test is made by whole abstract, to prevent leaks. §3 The dodging rewrites keep the content and match human text in length. §3 The paper expects a version like PNU + TTA to work best in practice. §3

check yourself

Why does it matter that PU + TTA and the supervised detector use the same classifier?

In the human-drift test, why are mirrors the known class instead of human text?

chapter 4

What they found

Shift 1: humanizers

As the dodger, they ran an autoresearch loop with Claude Code: the agent kept rewriting its instruction until the detector missed the output. §4.1 The result is an adversarial prompt. Then the detectors answered. The supervised detector was retrained on the new AI text with labels. The test-time detector got the new text only as part of its unlabeled pile. They ran three rounds. §4.1

guess first

Against text from the adversarial prompt, what share do you think the test-time detectors caught?

The part below opens after your guess.

Figure 2a from the paper: catching AI text from a humanizer §4.1 open full size

Figure 2a from the paper: catching AI text from a humanizer

Left, naive prompt: all four detectors catch nearly all the AI text.

With the adversarial prompt, the two test-time detectors still catch most of it.

The supervised detector and Pangram drop. Pangram catches 27% of it. §4.1

Right: three rounds of attack and retraining. Purple (PU + TTA) stays near the top. Orange (supervised) ends near half.

On the first adversarial prompt, the paper's test-time approach caught 93.3%. Abstract In the last round, PU + TTA had AI recall of 99%. §4.1 The supervised detector had 49.2%. §4.1

Who moves last §2

A supervised detector learns one line between labeled human text and AI text. Then it is released and stays fixed.

A humanizer rewrites the AI text until it lands on the human side of the fixed line. Against the paper's adversarial prompt, Pangram catches 27.0%. Abstract

A test-time detector treats the texts it receives as an unlabeled batch. It redraws its line after the shift and catches 93.3%. Abstract

A PU learning detector that never saw the dodging text was beaten too. §4.1

How hard the attack got, and how much unlabeled text is needed

Each round got harder for the dodger. The agent needed 10 candidate prompts in the first round and 35 in the last. B.5 While the agent searched for a prompt, the test-time detector did not adapt. Its unlabeled pile stayed the same until the prompt beat it. A real test-time detector would keep adapting, so the paper says the attack was likely easier than it would be in practice. §4.1

A small amount of dodging text in the unlabeled pile was enough. §4.1 Appendix D.1 varies the amount. D.1 TTT also adapted to a stream of human and dodging text. §4.1

Shift 2: new LLMs

The paper trained a supervised detector on mirrors from one LLM, then tested it on another. §4.2

guess first

A supervised detector was trained on Llama text. What share of gpt-oss-120b text do you think it caught?

50of 100 percent

The part below opens after your guess.

Figure 3 from the paper: train on one LLM, test on another §4.2 open full size

Figure 3 from the paper: train on one LLM, test on another

Each cell is the AI recall of a supervised detector. The diagonal is train and test on the same LLM: high.

Off the diagonal it drops. This row averages the cells where the test LLM was not the training LLM.

The two test-time detectors, which saw unlabeled text from the test LLM, stay high in every column.

On text from the training LLM, AI recall almost always exceeded 89%. §4.2 On other LLMs it fell. A detector trained on Llama 3.3 70B Instruct caught 55% of gpt-oss-120b text. §4.2 The test-time detectors came close to a detector trained on the test LLM. §4.2

Pangram and GPT 5.4

GPT 5.4 came out in March 2026. Soon after, Pangram flagged only 34.0% of its abstracts as AI or AI-assisted, two of the labels Pangram gives. The third is human. §4.2 After Pangram 3.3.2 came out in May 2026, it did much better. §4.2 The paper reads this as the supervised approach working as designed: wait for labeled data, then retrain. §4.2

Pangram itself reported near-perfect results on GPT 5.4 at the time. The paper says it is unclear why the two differ. §4.2 Appendix D.6 has the full comparison. D.6 The results across LLMs repeat on the RAID benchmark. §4.2

Shift 3: human writing drifts

There is no verified human text from after 2022 to test on. §4.3 So the paper pretends that LLMs arrived in 2010. Detectors train only on 2010 human text. They are tested on human text from 2010 to 2020, plus fresh mirrors of it, so the answers are known. §4.3

guess first

Liang et al. was trained on 2010 text, then run on human abstracts from 2020. What share do you think it called AI?

The part below opens after your guess.

Figure 4a from the paper: human writing drifts from 2010 to 2020 §4.3 open full size

Figure 4a from the paper: human writing drifts from 2010 to 2020

Left: the fixed supervised detector calls less and less human text human, from 85.9% to 73.4%.

The test-time detectors stay steadier. They end at 89.8% and 85.9%.

Right: Liang et al. trained in 2010 drifts upward. It finds more and more “AI text” in human abstracts.

PU + TTA stays near zero: its estimate does not drift.

The fixed supervised detector's human recall fell from 85.9% to 73.4% over the ten years. §4.3 The test-time detectors stayed steadier. §4.3 Liang et al. estimates the share of AI text in a collection. By 2020, it called 15.1% of human sentences AI, up from 2.3% at the start. §4.3

These are the costly errors: real human writing labeled as AI. §4.3

Prevalence estimation from 2020 to 2025 (Figure 4b)

The paper notes that Liang et al. is used to estimate AI use in many published studies. §4.3 For prevalence estimation, the bias of Liang et al. grew by 11.4% over the ten years. §4.3 The paper also trained on 2020 text and estimated AI use in 2023 and 2025. The supervised detectors found an increase over three times as large as PU + TTA found. §4.3 There is no answer key for 2025. The paper argues the true share is closer to the PU + TTA estimate. §4.3

check yourself

Why did the fixed detector's errors in the drift test fall on human text rather than AI text?

In the attack game, a PU learning detector that never saw the dodging text was beaten. What does this show?

chapter 5

What this does not show

The paper states these limits itself. §6

TTA learns from whatever texts people send. Could someone flood the unlabeled pile on purpose to move the line? The paper's attack holds the pile fixed and does not test this.

check yourself

Why can no detector win if AI text and human text end up with the same distribution?

Why is drift in human writing a harder problem than a new LLM?

chapter 6

Questions people ask

Short answers to common questions. Each one points back to the paper.

How do AI text detectors work?

Builders take human text written before 2022. An LLM rewrites each text, and the rewrite is a mirror. §1 A supervised detector detector learns to tell human texts from mirrors. §1 Once released, it stays fixed. Figure 1

Are AI text detectors accurate?

When test text matches training text, modern detectors such as Pangram are over 99% accurate. §1 When the text changes, they can fail. Pangram caught 96% of ordinary AI rewrites and 27% of rewrites made to dodge it. §4.1

Can AI text detectors be fooled?

Yes, in the paper's tests. A humanizer rewrites AI text to dodge a detector. §1 The authors had an agent search for such an instruction, and Pangram caught only 27% of the text it produced. §4.1

What is distribution shift?

It means the test text differs from the training text. Abstract The paper studies three kinds. These are humanizers, new LLMs, and changes in human writing. §4

What is test-time adaptation?

It means a detector learns from the texts it receives after release, without labels. §1 The paper's main detector, PU + TTA, caught 99% of the AI text in the last round of the attack. A supervised detector caught 49.2%. §4.1

keep it

This page has 10 check-yourself questions. The review page asks them again after 1, 3, 7 and 21 days, one at a time. You can also download them for Anki.

add these 10 questions to my review

chapter 7

Words to know

Hover or tap any underlined word on the page to see its entry. These are the terms this page uses, in the order it introduces them.

supervised detector (detector trained once on labeled text)
A detector that learns from texts whose answer (human or AI) is known, and then stays fixed after it is released.
"Deployed AI text detection approaches are supervised, relying on training-time access to labeled datasets of both human-written and AI-generated text." (Abstract)
mirror (AI rewrite of a human text)
Take a real human abstract and ask an LLM to rewrite it. The rewrite is AI text on the same topic.
"(b) generate AI-generated text themselves (“synthetic mirrors”) by querying an LLM to rewrite the human text." (§1)
also called in the paper: synthetic mirror
AI recall (share of AI text caught)
Of all the AI texts, the share the detector calls AI.
"AI Recall & Accuracy conditioned on AI-generated text" (§B.2)
also called in the paper: detection rate
human recall (share of human text called human)
Of all the human texts, the share the detector correctly calls human. When it drops, real people get flagged as AI.
"Human Recall & Accuracy conditioned on human writing" (§B.2)
distribution shift (change in the kind of text)
The texts a detector sees after release look different from the texts it was trained on.
"This approach is vulnerable to three types of distribution shifts that occur continually post-deployment, and for which labeled data is often unavailable: adversarial humanization, new LLMs being released, and unidentifiable temporal drift in human writing after widespread LLM usage." (Abstract)
out-of-distribution (unlike the training text)
Text that differs from the kind of text a model was trained on. In-distribution text is like the training text.
not defined in the paper
humanizer (rewriter that hides AI text)
A tool or prompt that rewrites AI text so a detector calls it human.
"adversarial humanization, in which users or humanizer services strategically rewrite AI-generated text to evade a published detector" (§1)
also called in the paper: humanization
continual distribution shift (text that keeps changing)
The change does not happen once. It keeps happening, and nobody hands the detector the new answers.
"These shifts share two features: (1) they continually occur, and so the task differs from one-time transfer learning; and (2) labeled test data may never be available or come with substantial cost." (§1)
unlabeled (with no known answer)
Texts whose answer, human or AI, nobody knows.
"Our insight is that while it may be hard to classify any single shifted text sample, detection becomes feasible given a cohort of unlabeled samples (i.e., a sufficiently large test-time batch)" (§1)
positive-unlabeled learning (learning from one known class and a mixed pile)
You know some texts are human. You have a pile that mixes human and AI. You learn to find what in the pile is not like the known human texts.
"A PU classifier uses known labels from one class, and an unlabeled mixture set" (§2)
positive class (known class)
The one class the detector has labeled examples of. In most tests here it is human text.
"We train such PU classifiers on one of the train-time data classes (either AI or human) as the known positive class and unlabeled data sampled from the test-time distribution." (§3)
test-time homogeneity (sameness of AI texts in a batch)
Many people use the same few models and tools, so the AI texts that arrive together tend to look alike.
"existing methods (supervised or otherwise) do not leverage test-time homogeneity in LLM text, which reflects algorithmic monoculture" (§1)
test-time adaptation (adapting to the texts it receives)
The detector keeps learning after release, from the texts people send it, even though nobody tells it which are AI.
"test-time adaptation (TTA), which adapts an existing classifier or re-trains a new classifier given unlabeled test-time data." (§1)
separable (can be split perfectly)
Some detector of the kind being trained could split the two groups with no mistakes.
"Thus, to make progress, we assume that H is separable from A ∪ h(A) by some classifier in the hypothesis class F" (§2)
PU + TTA (the one-class adapting detector)
The paper's main detector: labeled texts of one class only, plus the unlabeled texts it receives.
"We train such PU classifiers on one of the train-time data classes (either AI or human) as the known positive class and unlabeled data sampled from the test-time distribution. We also adapt (TED)^n to include labeled negatives from the source domain (i.e., a PNU learning method; similar in principle to Kiryo et al. [2017]). We refer to these methods as PU + TTA and PNU + TTA, respectively." (§3)
(TED)^n
A published positive-unlabeled learning method (Garg et al., 2021). It guesses how much of the pile is the known class, then trains on the rest as the other class.
"Primarily, we use the positive-unlabeled (PU) (TED)^n method of Garg et al. [2021], which works effectively in high dimensions." (§3)
PNU + TTA (the two-class adapting detector)
Like PU + TTA, but it also gets the labeled examples of the other class from training time.
"We also adapt (TED)^n to include labeled negatives from the source domain (i.e., a PNU learning method; similar in principle to Kiryo et al. [2017]). We refer to these methods as PU + TTA and PNU + TTA, respectively." (§3)
naive prompt (plain rewrite prompt)
An ordinary instruction to rewrite an abstract, with no attempt to fool a detector.
"otherwise, we use the prompting strategy defined in Liang et al. [2024] (we refer to this as the “naive prompt”)." (§3)
Test-Time Training
A test-time method by Sun et al. (2020). It updates the model on incoming text with a fill-in-the-blank task, without full retraining.
"For TTT, we pre-train the detection model using both a supervised and self-supervised loss function; at test-time only the self-supervised loss is used to update the model." (§3)
autoresearch loop (agent that keeps trying prompts)
A coding agent writes a prompt, sees which outputs the detector caught, and writes a better prompt, over and over.
"We use Claude Code to generate candidate prompts in an autoresearch loop" (§4.1)
adversarial prompt (detector-dodging prompt)
A rewrite instruction that was tuned until one particular detector calls its output human.
"For experiments involving strategic behavior, mirrors are produced using an adversarial rewriting prompt, optimized via an autoresearch loop, to specifically evade a given detection model" (§3)
prevalence estimation (estimating the share of AI text)
Not judging one text, but estimating what share of a whole collection is AI-written.
"For prevalence estimation (estimating the proportion of AI-generated text in a corpus)" (§1)
bias (estimation error)
The estimated share of AI text minus the true share. Zero is perfect.
"For prevalence estimation, we primarily use bias, the difference between estimated and true prevalence of LLM text." (§3)
Pangram
A commercial AI text detector, trained with supervised learning. The paper uses it as a strong baseline; versions 3.2 and 3.3.2 appear.
§1
DistilBERT
A small open language model. The paper fine-tunes it as the classifier inside both its supervised and its test-time detectors.
§3
the method of Liang et al.
A supervised method that estimates what share of a collection is AI text from word counts. Several studies use it.
"For prevalence estimation (estimating the proportion of AI-generated text in a corpus), Liang et al. [2024] propose a lightweight n-grams-based method." (§1)
Claude Code
Anthropic's coding agent. The paper uses it to write humanizer prompts.
§4.1
Llama 3.3 70B Instruct
An open model from Meta. One of the five LLMs in the new-model test.
§B.2
gpt-oss-120b
An open-weight model from OpenAI. It writes the AI text in the humanizer experiments.
§B.2
GPT 5.4
An OpenAI model released on March 5, 2026. The paper tested Pangram on it right after release.
§4.2
RAID
A public benchmark for AI text detection. The paper repeats its tests on it.
§3