Your Brain Doesn't Think in Sentences. Why Should an LLM?

Before you say a sentence, you usually already know what you mean. The words arrive afterwards, and sometimes they don't arrive at all: you hold an idea you can't phrase yet. That gap between a thought and its wording is exactly what a language model lacks.

Published on November 21, 2026

Your Brain Doesn't Think in Sentences. Why Should an LLM?

An LLM runs one loop: predict the next token. Deciding what to say and deciding how to say it happen in the same forward pass, one token at a time. In humans, psycholinguistics treats these as separate stages.

A recent paper, Next-Concept Modeling (NCM), generates text coarse to fine: the gist first, the exact wording last. It is not human-like thinking, and its authors don't claim it is. But it breaks the token-by-token assumption in a useful way. This post compares how LLMs and humans think and remember, looks at what NCM actually does, and argues for where the idea could go next.

How an LLM works: thinking and wording are one loop

An autoregressive LLM does one thing: given the tokens so far, produce a distribution over the next token, sample one, append it, repeat. Every step runs the whole network.

There is no place in this loop where an idea exists apart from its phrasing. Whatever "reasoning" happens must be completed inside a single pass, then committed as a token. Once a token is emitted, the model can't revise it; it can only build on it.

Chain-of-thought is the workaround. To think longer, the model has to write its thinking out as sentences, because sentences are the only scratch space it has. That is a telling design: the model's reasoning medium is its output language.

The cost shows up in the parameters. The same weights must store grammar, style, world knowledge and reasoning patterns, all entangled, because nothing in the architecture asks them to be separate.

How LLM memory works: three layers, no consolidation

An LLM has three places to keep information, and none of them behaves like human memory.

  • Weights (parametric memory). Facts and skills compressed into parameters during training. It is fuzzy, hard to edit, and frozen at deployment.
  • Context window (working memory). Everything the model can attend to right now. It is large, but flat: every token is equally "present", and it is wiped when the session ends.
  • External stores (RAG, vector databases, agent memory files). Retrieved text pasted back into the context. Useful, but bolted on. The model doesn't build these memories; the surrounding system does.

The dotted arrow is the important one. What the model reads and does during a session never becomes a lasting abstraction in the weights. There is no step that turns experience into concepts.

How humans think and remember: concept first, words later

The standard psycholinguistic account of speaking, Willem Levelt's model, has three stages. A conceptualizer decides what to mean. A formulator picks words and builds grammar. An articulator produces sound or text. Grammar is a late stage that is mostly automatic. You don't plan the subject-verb agreement of a sentence; you plan the point.

A fair caveat: people do have inner speech, and language shapes how we categorize and remember. The claim here is narrower. Concepts are primary, and wording is a separate step that can fail independently. The tip-of-the-tongue state, where you know exactly what you mean and can't retrieve the word, is the classic evidence.

Human memory is also structured differently from an LLM's:

  • Working memory holds a handful of items, not a whole transcript.
  • Episodic memory stores specific events. Semantic memory stores general knowledge. Procedural memory stores skills.
  • Consolidation, helped by sleep, gradually turns episodes into abstractions. You forget the details of a hundred meetings and keep a concept of how your manager thinks.
  • Recall is reconstruction. You rebuild a memory from concepts plus cues, which is why it is flexible and also why it can be wrong.

Side by side

The differences come down to where the abstraction happens and whether anything is learned after training.

LLMHuman
Unit of thoughtTokenConcept
Thinking vs. wordingFused in one forward passSeparate stages
Scratch space for reasoningIts own output textNot language-bound
Learning after trainingNone, or bolted onContinuous, consolidated
RecallPattern completionReconstruction from concepts
Memory structureWeights + flat contextWorking, episodic, semantic, procedural

Enter NCM: generate the gist first

Next-Concept Modeling (Kumbong et al., Stanford and Together AI, October 2026) replaces left-to-right generation with a coarse-to-fine hierarchy. It has two stages.

  1. A tokenizer (a multiscale residual VQ-VAE with nested dropout) turns text into K levels of discrete codes. Level 1 is a short sequence; each later level is longer; the last is full token length. Nested dropout forces the early levels to carry real information.
  2. A transformer then generates one level per step. All codes within a level are predicted in parallel, conditioned on the reconstruction built from the earlier levels. The tokenizer's decoder turns the finished reconstruction back into text.

What "concept" means here. The paper defines it narrowly: the information carried by the coarse levels. It is a learned compressed sketch of the whole text, not a human idea that exists independently of its wording. The name is suggestive; the mechanism is more modest.

What the results show. All figures below are from the paper, at roughly 100M-parameter generators.

  • On LM1B, NCM beats the strongest diffusion language model baseline (DUO) on generative perplexity, 93.05 against 97.30, using 12 function evaluations against 128 steps, about 21× faster.
  • On OpenWebText it uses 32 evaluations against 1,024 and is about 10× faster, with slightly worse perplexity (83.12 against 77.69).
  • It needs about 4 to 5× fewer training token positions than the baselines (13.4B against 66B on LM1B).

The Sudoku result is the interesting one. A 25M-parameter NCM solves 86.1% of hard puzzles, against 81.3% for the best baseline. Because every level sees the whole board, the model can plan globally and revise earlier coarse decisions. Press coverage of the paper reports that a same-size autoregressive model solves only about 1% of hard boards (TechNews). That is a computational advantage of gist-first generation, not only an analogy.

Caveats. The speed baselines are diffusion and flow models, not optimized autoregressive serving. The main models are around 100M parameters, and the scaling study (up to 725M on LM1B) is described as preliminary.

Where this sits in a wider trend. Meta's Large Concept Model predicts sentence-level embeddings in a language- and modality-agnostic space, but its concepts are human-defined sentences. DLCM learns where to concentrate reasoning and separates token-level capacity from concept-level reasoning capacity. NCM's distinction is that its levels are learned from data.

Where this could go: separating the reasoner from the writer

This section is a hypothesis, not a result. NCM's published gains are speed and training efficiency. The paper makes no claim about parameter counts or about human cognition, and one transformer generates every level.

The hypothesis is this: if the coarse levels carry the meaning and the fine levels carry the phrasing, the two jobs need different amounts of capacity. Most parameters could sit in a concept-level reasoner. The wording stage could be a small decoder, and it could be swapped per language or output format.

There is indirect support for the direction. Meta's Large Concept Model shows that a concept-level core can sit between swappable encoders and decoders. DLCM argues that compute should concentrate where information is dense, instead of being spent uniformly per token.

For the hypothesis to hold, three things would have to be true, and each can be tested:

  1. The coarse levels are largely language-independent, so one reasoner can serve many decoders.
  2. A fine-level decoder much smaller than the reasoner loses no quality.
  3. At matched compute, a concept-reasoner design matches or beats an autoregressive model of the same total size.

None of these is shown by NCM today. They are the experiments this direction invites.

The missing piece: memory that lives in concept space

Concept-level generation does not, by itself, fix memory. NCM is a text generator; it has no memory system beyond its context. But the two ideas fit together, and that is the part I find most promising.

Today's agent memory is keyed by text: chunks of past conversation, embedded and retrieved by similarity. That is the wrong index. Humans don't recall the exact sentence they heard; they recall the concept and rebuild the wording.

A human-like memory system built on a concept-level model would need three things:

  • Concept-indexed storage. Memories saved as concept codes, not raw text, so retrieval matches meaning across languages and phrasings.
  • Consolidation. A background process that merges many episodes into fewer, more abstract entries, as sleep does for people.
  • Episodic to semantic distillation. Specific events gradually becoming general knowledge, with the details allowed to fade.

None of this exists in NCM. It is the natural next question once generation and reasoning are no longer the same operation.

Open questions and limits

  • Does the coarse level lose detail the fine level can't recover? The paper's tokenizer reconstructs text almost exactly (99.99% on OpenWebText, 98.79% on WMT14 translation), but that is a property of the tokenizer, not of generation quality at scale.
  • Is a learned "concept" human-like, or just a compression level? Nothing in the paper says it is. Treat the name as a metaphor until someone probes what the coarse codes encode.
  • Fixed depth. The hierarchy has a fixed number of levels and fixed per-level lengths, so even a short continuation walks the whole hierarchy. The authors suggest input-dependent schedules.
  • Separate training. The tokenizer is trained apart from the generator. End-to-end training is listed as future work.
  • Scale. Main results are around 100M parameters. Whether the gains hold at billions is unknown, and the paper notes occasional artifacts where coarse levels hand off.
  • Evaluation. We lack a clean way to measure concept-level reasoning separately from fluency, which is exactly the distinction this post is about.

Closing

Go back to the moment before you speak, when you know what you mean and the words haven't come. That moment is cheap for you and impossible for a token-by-token model. NCM doesn't recreate it, but it does show that generating the gist first is workable, fast, and in at least one task (Sudoku) better than going left to right.

The next step after scaling may not be bigger models. It may be models that decide what to mean first and phrase it second.

Sources: Next-Concept Modeling (arXiv 2610.14162) · Hazy Research NCM repo · Large Concept Models (Meta, arXiv 2412.08821) · DLCM (arXiv 2512.24617) · TechNews coverage of NCM