Mechanism 2: Factual priming
When we analyze the natural reasoning traces generated for simple factual questions, we notice a common pattern. The models aren’t writing out logical proofs; they are surfacing related facts.
In human cognition, there is a concept known as spreading activation, where processing a specific concept primes related concepts in semantic memory, making them easier to retrieve. We hypothesize that language models exhibit a similar generative self-retrieval mechanism, which we call factual priming. By generating facts topically related to the question, the model builds a contextual bridge that facilitates the retrieval of the correct answer.
To test hypotheses, we extract just the concrete facts from the model’s reasoning traces, applying strict filtering to strip away any filler text, search plans, or explicit mentions of the final target answer. We then isolate the effect of the recalled facts, and show that conditioning on a short list of recalled facts recovers most of reasoning’s gains and helps even when reasoning is OFF.
💸 Earn Instantly With This Task
No fees, no waiting — your earnings could be 1 click away.
Start Earning