What question did this study set out to answer?

This research aims to assess whether Dynamic Embedding Injection can effectively replace token-level RAG in language models for question answering tasks.

April 14, 20260 citationsOpen Access

View Full Paper

Can Mid-Layer Embedding Injection Replace Token-Level RAG? An Empirical Analysis

KKim

Key Points

This research aims to assess whether Dynamic Embedding Injection can effectively replace token-level RAG in language models for question answering tasks.
Evaluated Dynamic Embedding Injection across three QA datasets: SQuAD v2, NQ-open, TriviaQA.
Tested two model architectures: Qwen3-1.7B Transformer and Qwen3.5-9B Gated DeltaNet hybrid.
Analyzed reductions in cross-entropy loss and context token usage compared to token-level RAG.
Dynamic Embedding Injection reduced cross-entropy loss by 14.9% to 62.9% over no-retrieval baselines.
Used approximately 88% fewer context tokens than token-level RAG.
A persistent gap of ~65% with token-level RAG suggests limitations in residual-stream injection.

Abstract

We investigate Dynamic Embedding Injection (DEI), a method for injecting retrieved document knowledge into language model intermediate layers as dense embeddings rather than context tokens. Across three QA datasets (SQuAD v2, NQ-open, TriviaQA) and two model architectures (Qwen3-1.7B Transformer, Qwen3.5-9B Gated DeltaNet hybrid), DEI reduces cross-entropy loss by 14.9–62.9% over no-retrieval baselines while using ~88% fewer context tokens than token-level RAG. A persistent ~65% gap with token-level RAG remains, consistent with an information-capacity bottleneck in residual-stream injection that lightweight LoRA adaptation did not close within our training budget. Code and experiment logs: https://github.com/2264K/dei

AIに質問

Bookmark

View Full Paper

Cite This Study

Kim (Sun,) studied this question.

synapsesocial.com/papers/69ddd9cae195c95cdefd726e https://doi.org/https://doi.org/10.5281/zenodo.19544120

AIに質問

Bookmark

View Full Paper