We test whether hierarchical text retrieval improves over flat dense retrieval on multi-hop scientific document QA. On n = 500 multi-hop DocHop-QA questions, HiRAG using the same encoder at the document and section stages ties flat dense retrieval on every metric we measured (paired Wilcoxon p = 0. 79; Cohen's d = -0. 016, a negligible effect; 345 / 500 questions receive bit-identical judge scores). We diagnose the cause as a 0. 15 absolute gap between document-level and section-level recall@5 that is invariant across three independent retriever families (jina dense, OpenAI dense, BM25 sparse) and trace it to a lexical-overlap bias: gold sections that dense retrieval hits at top-5 have +35% higher question-section Jaccard than gold sections it misses (Welch t = 13. 15 on n = 1, 985 query-section comparisons, p << 0. 001). A second benchmark converges. On M3DocVQA (n = 464 multi-page PDFs, an entirely different modality and domain), we replace a heuristic typed-edge graph-traversal policy (D1) with an LLM-guided per-hop traversal policy (LG1, claude-sonnet-4-6 routing each hop) and find LG1 is statistically significantly worse under both answer LLMs we tested: Δ = -5. 17 pt under qwen3. 6-plus (paired McNemar p = 0. 015) and Δ = -9. 05 pt under claude-sonnet-4-6 (χ²cc = 14. 746, p = 0. 000104). A cross-LLM decomposition rules out the "qwen is the weak link" reading: a stronger answer LLM rescues D1 (+5. 39 pt, p = 0. 006) but cannot rescue LG1 (+1. 51 pt, p = 0. 54, n. s. ). We frame the result as a statistically significant cross-dataset null result for retrieval-side re-architecting, held under two answer LLMs. Code & data: https: //github. com/apertis-ai/multihop-doc-qa
Yu‐Ting Lee (Fri,) studied this question.