PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
April 12, 20260 citationsOpen Access

Indexed Key Retrieval from LoRA Adapters for Continual Learning

View Full Paper
TPTobias Preusser

Key Points

  • The aim is to develop a mechanism that encodes knowledge into LoRA adapters for effective recall.
  • Mechanism designed for encoding personal knowledge into LoRA adapter weights
  • Utilized a SimHash fingerprint registry for detection of errors
  • Validated performance across three different AI model families on a single GPU
  • Experimented with incremental training while monitoring recall of older keys
  • Achieved successful recall of 100 keys with a mechanism
  • Identified significant catastrophic forgetting with 0/40 old keys retained after 5 cycles of incremental training
  • Extended experiments maintained 100% recall even when scaling to over 550 keys
  • Documented both successes and failures in knowledge retrieval strategies

Abstract

This paper presents a mechanism for encoding personal knowledge into LoRA adapter weights with per-fact addressable recall. Each fact is assigned a unique string key, and the adapter learns to return the corresponding QA pair when prompted — enabling enumeration, verification, and lifecycle management of stored knowledge. A SimHash fingerprint registry provides hallucination detection at 8 bytes per key. The paper validates the core mechanism to 100 keys across three model families (Qwen 2.5 3B, Gemma 2 9B, Mistral 7B) on a single GPU (8 GB VRAM, QLoRA 4-bit), and reports both successes and revealing failures: incremental training without replay causes catastrophic forgetting (0/40 old keys after 5 cycles), and neither additive composition nor weight merging preserves indexed recall across adapters. Extended experiments including scaling to 550+ keys with 100% recall maintained, natural language generalization tracking, and a working home deployment are documented in the project repository: https://github.com/tiberius1701d/paramem

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Tobias Preusser (2026) studied this question.

synapsesocial.com/papers/69db38534fe01fead37c696bhttps://doi.org/10.5281/zenodo.19502522
Ask AI
Helpful
Bookmark
Share
View Full Paper