Key points are not available for this paper at this time.
v2 (2026-05-14) integrates Paper 14's extended-range empirical confirmation: §6 now cites the 11-checkpoint reactance-cliff dose-response (ep₀ to ep₃00, 5× standard training depth on Qwen2. 5-7B-base + paraphrase-FT), which rises monotonically from −2. 45 to +13. 96 log-units with no plateau — the same substrate-mechanism observed on a different metric, extending the empirical reach by half an order of magnitude in training-depth. §7. 5 adds a "behavior-signal decoupling" paragraph documenting that match-accuracy plateaus around 90% after roughly 100 epochs while substrate-state reactance-cliff continues to grow from +9. 68 to +13. 96 log-units — practical implication: behavioural performance and substrate-state divergence track distinct quantities. References updated: Paper 14 entry added; Paper 13 entry corrected to canonical (2026n) + DOI 10. 5281/zenodo. 20059877. Why do fine-tuned language models hallucinate more confidently than their in-context-learning counterparts on the same knowledge? Two well-established techniques add new knowledge to a language model: keeping it in the prompt — in-context learning (ICL; Brown et al. 2020) — or training it into the weights, here via low-rank adapters (LoRA; Hu et al. 2021). These are usually framed as alternatives along a deployment-cost axis. We show they are not interchangeable. Mechanism. We trace the difference to a substrate-mechanism: each backward pass under cross-entropy loss amplifies the winning route asymmetrically and presses alternatives below the noise floor; the depth of this compression scales with how many gradient passes the substrate has absorbed. ICL preserves the model's calibrated distribution over candidate answers because no weight update has compressed it. FT compresses it as a structural consequence of cumulative gradient pressure, regardless of training-data content. The architecture-level distinction this produces between ICL and FT maps onto the working-memory / long-term-memory distinction familiar from cognitive science: ICL operates in working-memory mode (forward-only computation, full distribution recomputed per turn, alternatives accessible) ; FT in long-term-memory mode (consolidated trace, alternatives compressed). The mechanism generalises a previously RLHF-specific finding (Pødenphant Lund 2026b §3) to all weight-update training — including plain LoRA fine-tuning on innocuous factual data. Empirical findings. Across three experiments on a 47-fact invented knowledge domain (Zorbetik), Qwen2. 5 base models at 3B and 7B scales, and LoRA fine-tuning budgets from 5 to 100 epochs plus a paraphrase-augmented variant: ICL outperforms LoRA-FT on cloze retrieval by 16–28 percentage points across capacity scales; FT-trained models actively degrade application accuracy below the no-context baseline; the per-token competing-routes signal collapses monotonically with cumulative gradient passes — log (CRₚos0) climbs from 5. 46 (ICL) to 17. 85 (raw FT, 30 epochs) to 21. 12 (paraphrase-augmented FT, 30 epochs) ; entropy at position 0 collapses from 0. 32 (ICL) to ≈0. 00 (any FT regime), independent of training-data variation. v2 adds Paper 14's 11-checkpoint extended-range cross-paper confirmation on reactance-cliff amplitude (the same mechanism observed on a different observable across ep₀ to ep₃00). Applied consequences. Why FT-trained models hallucinate more confidently: alternative routes have been compressed beyond reach. Why agentic systems can't reliably represent uncertainty when built on FT: the substrate-level signal that would carry uncertainty is gone. What the RAG-vs-FT debate looks like at the substrate level: RAG operates in ICL-mode and preserves calibration; FT compresses it as a structural consequence. How long-context agentic conversations (Claude Code, Cursor, multi-turn agents) inherit ICL's calibration properties for free: each turn re-evaluates the full context with no weight update. And how concrete hybrid ICL+FT memory architectures can approximate biological memory's two-system structure with bounded context-window cost. Caveats. FT is tested as LoRA only (full-parameter FT untested) ; single random seed; two model sizes (3B, 7B) ; one invented domain. Paraphrase-augmentation in Experiment 3 confounds data-variation with cumulative gradient-pressure (≈10× more gradient steps) ; a clean compute-matched comparison is left to follow-up. The qualitative ICL-vs-FT direction is robust within these scope conditions. Companion papers in the Friction Theory series: Paper 0 (BFT master): 10. 5281/zenodo. 19462500 Paper 1 (Friction Theory substrate): 10. 5281/zenodo. 20012655 Paper 2 (Capacity Scaling, the empirical companion): 10. 5281/zenodo. 20013491 Paper 3 (Friction-Guided Inference): 10. 5281/zenodo. 20014122 Paper 10 (Race-Architecture Physics): 10. 5281/zenodo. 20014568 Paper 13 (Operational Friction Theory): 10. 5281/zenodo. 20059877 Paper 4, Paper 4B, Paper 6, Paper 14 (Logic as Reactance) — in preparation Data and code. Per-token logprob datasets, fine-tuning notebooks, and analysis scripts share Paper 2's companion repository: https: //github. com/tplund/friction-theory-p2-capacity-scaling (CC BY 4. 0).
Building similarity graph...
Analyzing shared references across papers
Loading...
Tomas Pødenphant Lund (Thu,) studied this question.
www.synapsesocial.com/papers/6a080a29a487c87a6a40c072 — DOI: https://doi.org/10.5281/zenodo.20187352
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:
Tomas Pødenphant Lund
Aarhus University
Building similarity graph...
Analyzing shared references across papers
Loading...