Do LLMs Think? Next-Token Prediction and Generation Settings

A fluent reply is not evidence that a model has thought. At its core an LLM performs next-token prediction. Conversation-like and reasoning-like output emerges from the autoregressive loop, sampling, and post-training layers stacked on top of pretraining.

📑Table of Contents

1. Next-token prediction is a probability chain, not understanding
2. Transformers handle long context with parallel attention
3. Pretraining alone does not make an assistant
4. Different answers to the same question are usually sampling
5. A field checklist for output that looks like thinking
6. FAQ
7. Summary

How this guide is organized

In AI operations, a confident tone and checklist-shaped prose are easy to mistake for a verified conclusion.

This guide first separates official primary sources from independent explainers, then converts that distinction into a practical field checklist.

---

Official sources used here

• Official: Transformer paper
• Official: InstructGPT
• Official: Hugging Face GenerationConfig

Reader next action: on one recent LLM output, open the product or API generation settings, record do_sample/temperature/top_p, then run the six-item fluency-versus-thinking checklist. Re-read as a next-token chain, confirm sampling vs greedy, lower temperature if replay is needed, separate SFT/RLHF from proof of understanding, verify facts and quotes outside the model, and keep the pass condition external.

---

Next-token prediction is a probability chain, not understanding

Each generation step selects the next token from a probability distribution. That step is not a separate meaning engine or a truth judge.

How the generation loop is decomposed

Grune and Speculative Chic break the loop down as:

1. Tokenize the input
2. Embed each token
3. Apply self-attention over the other tokens
4. Predict the next token
5. Append the chosen token and repeat

---

Training versus how the output looks

School of Web describes training as hide-and-guess over internet-scale text. What looks like knowledge is often co-occurrence statistics. What looks like conversation is that loop running quickly.

---

Completions that look like answers

Mahdi Naser Moghadasi uses a simple prompt: “The capital of France is” continuing as “Paris.” The completion looks like question answering or thought. The mechanism is still placing a high-probability next token.

Independent essays are not official loss-function definitions. Maintain the following clear split.

• Official claims stay on the Transformer paper and the InstructGPT abstract
• Explainers are a reading of the mechanism
• “Reads as if it understands” and “has been verified” are different checks

---

Transformers handle long context with parallel attention

The 2017 Transformer made next-token computation practical by attending over the full sequence instead of stepping through an RNN.

Official paper numbers

Vaswani et al., Attention Is All You Need (arXiv:1706.03762) reports:

• Attention only; no recurrence and no convolution
• 28.4 BLEU on WMT 2014 English–German, more than 2 BLEU above the previous best ensembles
• 41.8 BLEU on WMT 2014 English–French with a single model after 3.5 days on eight GPUs

---

How to read parallel attention

Grune and Speculative Chic treat this paper as the core of ChatGPT-class models and describe the decoder loop: predict one word, append it, predict again. Seeing context in parallel is not proof of understanding. It is a device for computing the next-token conditional distribution more stably.

Use BLEU only as follows:

• It is a machine-translation score
• Do not use it as a pass/fail for your internal chatbot or document Q&A
• The paper showed that attention alone could raise translation quality; it did not show that the model became a thinking engine

---

Pretraining alone does not make an assistant

Next-token pretraining simply continues text. Answering instructions is a later objective achieved through supervised fine-tuning (SFT) plus RLHF.

The two InstructGPT stages

Ouyang et al., InstructGPT (arXiv:2203.02155) states that making language models larger does not automatically make them better at following user intent. The official procedure is:

1. Collect demonstrations of desired behavior and supervised-fine-tune the model
2. Collect human rankings of outputs and further fine-tune with RLHF

Human evaluation, in brief:

• Raters preferred 1.3B InstructGPT over 175B GPT-3
• That is about 100× fewer parameters
• The abstract reports better truthfulness and less toxicity
• It still notes that the model makes simple mistakes

---

The pretraining-only failure mode

Harys Dalvi shows the pretraining-only failure mode: a prompt such as “Write an article about…” is continued as more prompt text rather than answered. Instruction finetuning comes first, then RLHF. That second-stage objective is not next-token loss.

---

Alignment is not a thinking engine

Polite, on-topic answers are an alignment result, not proof that the model became a reasoning engine. These numbers are 2022 GPT-3-era human evaluations. They do not license claims about any vendor’s unpublished 2026 training recipe.

If you treat instruction-following as understanding, the model will keep producing confident errors. Abstention and external verification belong in a separate design, covered in why LLMs avoid saying they do not know.

---

Different answers to the same question are usually sampling

A changed reply is rarely evidence that the model changed its mind. It is usually a different draw from the next-token distribution, shaped by temperature and truncation.

Hugging Face defaults

Hugging Face GenerationConfig and generation strategies (v4.46.3) document these defaults when unset:

Setting	Default if unset	What it means in operations
do_sample	False (greedy)	Always take the top token. Fine for short text; repeats on long output
num_beams	1	Above 1, keep high-probability sequence hypotheses
temperature	1.0	Sharpens or flattens next-token probabilities

---

Length and truncation defaults

Setting	Default if unset	What it means in operations
top_k	50	Keep only the top k tokens
top_p	1.0	If below 1, keep the smallest set whose mass is at least top_p
max_new_tokens	(must set)	Generated length excluding the prompt; prefer this over max_length

Source: Hugging Face GenerationConfig, generation strategies v4.46.3 (as of August 2026)

---

Temperature versus hallucination

Qiita’s Sakai_path and School of Web treat sampling as follows:

• Generation samples after softmax
• Greedy is stable but monotonous
• High temperature admits rare tokens and can collapse into incoherence
• There is no separate truth judge, so statistically plausible tokens can still hallucinate

For reproducible work, inspect the product or API do_sample, temperature, and top_p first. Lower temperature or move toward greedy, then verify facts outside the model. Setting temperature to 0 does not delete high-probability errors.

---

A field checklist for output that looks like thinking

Fluency, procedure-shaped lists, and confident tone are not verified conclusions. Split appearance from mechanism, then run one recent LLM output through the field procedure.

Field procedure for one recent output

On one recent LLM output, open the product or API generation settings and run these steps in order.

1. Record do_sample, temperature, and top_p
2. Re-read the output as a next-token chain, not as thought
3. Confirm sampling vs greedy; lower temperature if replay is needed
4. Separate SFT/RLHF instruction-following from proof of understanding
5. Verify facts, numbers, and quotes outside the model
6. Do not let the model grade itself; keep the pass condition external

This is the six-item fluency-versus-thinking checklist.

---

Appearance versus mechanism

What it looks like	What it usually is	Check first
A conversation	Autoregressive append of next tokens	Is there retrieval or tool execution beyond the generate loop?
Thinking	Scaled next-token prediction plus post-training	Does a separate module judge truth? Usually no

---

Instruction-following and confidence

What it looks like	What it usually is	Check first
Following instructions	SFT / RLHF, not the pretraining objective	Base pretrained model or instruction-tuned product?
Slightly different each time	temperature / top-p / sampling	do_sample and temperature; greedy is nearly fixed
Confidence	A chain of high-probability tokens	External sources, a replay path, and failure conditions

Source: operational synthesis of the Transformer / InstructGPT / Hugging Face docs plus Grune, Dalvi, Mahdi, School of Web, and Qiita (as of August 2026)

---

Checklist for one recent output

Checklist — apply once to the latest LLM output:

• ✓ Re-read the output as a next-token chain, not as thought
• ✓ Open the API or UI and record do_sample, temperature, and top_p
• ✓ If you need replay, lower temperature or move toward greedy
• ✓ Separate instruction-following (SFT/RLHF) from proof of understanding
• ✓ Check facts, numbers, and quotes outside the model
• ✓ Do not let the model grade itself; keep the pass condition external

---

Chain-of-thought text is still tokens

Even when a product shows chain-of-thought text, that text consists of generated tokens. Do not operate as if the model has awareness or grounding.

If you run an agent loop, first separate instructions, guards, and verification.

---

FAQ

Q1. If it is only next-token prediction, why can it translate or do math?

The official Transformer reports 28.4 / 41.8 BLEU on translation benches. Mahdi treats later abilities as emergent from scaling the same objective. Capability and “it is thinking” are different judgments.

---

Q2. Does a larger model become a mind?

InstructGPT’s abstract says scale alone does not fix intent-following. A 1.3B instruction-tuned model was preferred over 175B GPT-3. Keep scale and alignment as separate axes.

---

Q3. Does temperature 0 remove lies?

No. Temperature sharpens a distribution; it does not judge truth. Greedy decoding still emits high-probability errors. School of Web attributes hallucination to the missing truth judge.

---

Q4. Are ChatGPT-class products only next-token models?

Pretraining is next-token at the core. Assistant behavior is added later by SFT/RLHF, which is the shared claim of InstructGPT and Dalvi. Do not assert unpublished 2026 vendor recipes.

---

Q5. What should an operator do first?

On one recent LLM output, open the product or API generation settings, record do_sample/temperature/top_p, then run the six-item fluency-versus-thinking checklist. Re-read as a next-token chain, confirm sampling vs greedy, lower temperature if replay is needed, separate SFT/RLHF from proof of understanding, verify facts and quotes outside the model, and keep the pass condition external.

---

Related articles:

• 13555
• 13550

Summary

• Next-token prediction is not an understanding module. Fluency is how a probability chain looks
• The Transformer made that computation practical with parallel attention; BLEU scores translation, not thought
• Assistant behavior is a later SFT/RLHF objective, not autocomplete
• Answer drift is often a sampling setting
• Next action: on one recent LLM output, open the product or API generation settings, record do_sample/temperature/top_p, then run the six-item fluency-versus-thinking checklist. Re-read as a next-token chain, confirm sampling vs greedy, lower temperature if replay is needed, separate SFT/RLHF from proof of understanding, verify facts and quotes outside the model, and keep the pass condition external