A Single Direction Separates Computation from Retrieval in Transformer Hidden States

Key Points

• The study explores how a linear direction in transformer hidden states can separate computation tasks from retrieval tasks.
• Analyzed hidden states in transformer models across different subjects like physics and geography.
• Evaluated the model’s ability to distinguish recall from reasoning using area under the curve (AUC) metrics.
• Investigated classification of reasoning and factual errors among incorrect answers in MMLU.
• Validated findings across twelve models with varying architecture sizes from 70 million to 30 billion parameters.
• Achieved AUC scores ranging from 0.996 to 1.000 in distinguishing recall from reasoning.
• Generalized classification of reasoning errors versus factual errors with AUC scores from 0.878 to 0.951.
• Demonstrated that the distinguishing direction is nearly orthogonal to correctness metrics.
• Detection efficiency achieved through a single dot product per prompt, eliminating the need for model training or modifications.

Abstract