An October 21 preprint reports that several recent language models use English constructions associated with greater reading effort more often than comparison texts do. The authors examined model replies across conversation, coding and essay tasks, then ran a separate training experiment to test one possible reason for the pattern. Their measurements make a useful distinction for anyone reviewing AI-written explanations: a sentence can convey the right information while making its reader work harder to recover it.

The big change

  • What changed: The study puts measurable language features behind a familiar complaint about dense AI prose. Its strongest comparisons concern particular models and tasks, rather than a single score for how understandable AI text is.
  • Why it matters: People reviewing model-written explanations can look beyond factual correctness to how a sentence packages its facts. Rare word sequences, long links between related words and compressed clauses can add reading work even when the message survives a paraphrase.
  • What to watch: The training experiment shows that rewarding shorter answers can change a small model's grammatical choices. Whether that pressure explains the language of commercial models remains open because their training histories were not experimentally varied here.

What the researchers counted

Thomas Hikaru Clark, Matias B. Martin and Kyle Mahowald analyzed output from 14 models. Twelve completed simulated chat conversations, nine completed coding sessions and all 14 wrote essays. The nine frontier and near-frontier models highlighted in the abstract are a subset of that full collection. The chat and coding exchanges used a simulated user; the essay prompts came from the PERSUADE 2.0 student-writing corpus. The researchers analyzed final replies, excluding code blocks and other material that would distort the language measures.

The measures are clues about processing effort, not scores from people reading these particular replies. They include uncommon neighboring word pairs; the distance between words that depend on each other grammatically; three or more nouns stacked together; verbs made from nouns; and inanimate subjects paired with verbs usually used for animate subjects. The paper also counts object relative clauses, especially those that omit an optional “that.” In “the book the professor wrote,” for example, the reader must work out how “book” relates to “wrote” without that extra cue. That phrase is an example from the paper's experimental materials.

The authors compared model output with roughly matched human corpora where possible: subtitles for chat, StackExchange for coding and selected student essays for the essay task. They also sent each model sentence through translation into Japanese and back into English. This produced a paraphrase paired with the original sentence, allowing a comparison of wording while keeping much of the content similar. Translation can simplify language in its own right, so the paired result does not turn the paraphrase into a perfect human benchmark.

Across most frontier and near-frontier models, original replies had more rare word sequences than their translated paraphrases. The authors also report higher rates of several compressed constructions, including noun-derived verbs, inanimate subjects with animate verbs and relative clauses without “that.” The pattern was less pronounced for essays than for chat and coding, and individual models differed substantially. Llama and the Olmo base and instruct models changed least under paraphrasing; Olmo Think looked more like the frontier group. Claude Opus 5.5 scored lower than Opus 5 on 19 of 21 difficulty-linked comparisons in the paper, a result that cuts against a simple claim that every newer model is becoming harder to read.

What the length penalty establishes

The researchers then fine-tuned Qwen2.5-1.5B-Instruct, a much smaller model, on a controlled task: turn structured event descriptions into one English sentence. A reward favored a correct answer and, in some runs, deducted points for extra tokens. The training used group relative policy optimization, or GRPO, across 24 runs that varied the length penalty, a constraint on how far the model could drift from its starting behavior, and the random seed.

With a length penalty, the model increasingly chose shorter versions of constructions used in training, including clauses without an optional “that.” The preference also appeared in two held-out constructions involving “that.” Those tests show some transfer beyond the exact training items. For four more distant constructions, only the possessive form shifted with the penalty, and only in runs without the constraint on drift. The passive and purpose-clause forms did not shift; the quantifier form was already near its ceiling before training.

This is evidence that a reward for brevity can influence grammatical choices in this controlled setting. It does not identify the cause of the observed patterns in the nine commercial or near-frontier models. Their training objectives were not altered in the study, and the authors say they could not causally examine post-training at that scale.

Reading the result carefully

The study measured English text from a limited set of models, conversation starters and coding repositories. Its human corpora are useful reference points, but subtitles, developer posts and selected high-scoring student essays are not identical to the model tasks. Some feature detections also relied on an automated classifier, and the paper's multiple statistical comparisons were not adjusted for their number.

Most of all, no participants were asked to read the sampled model replies and report comprehension, reading time or mistakes. Earlier psycholinguistic work links the selected features to processing costs; this preprint shows where those features occur in its corpus. It leaves the size of any real comprehension effect, and the role of hidden frontier-model training decisions, for later work. For now, the practical reading is narrower and more useful: when an AI explanation feels dense, inspect the sentence's structure as well as its factual content.

Sources & further reading