What question did this study set out to answer?

The study aims to improve the performance of language models by using specialized components that communicate through tensor signals.

February 27, 2026Open Access

Micro-AGI: Emergent Intelligence from Networks of Small Language Models

Key Points

The study aims to improve the performance of language models by using specialized components that communicate through tensor signals.
Developed Micro-AGI architecture with LoRA adapters sharing a frozen language model.
Implemented a router network to activate cognitive specialists based on input characteristics.
Conducted five iterations of performance testing to compare routing-augmented responses against baseline.
Routing-augmented responses achieved scores up to +1.6 points higher than baseline.
Identified that the language model head compresses detailed internal representations into token probabilities.

Abstract

We present Micro-AGI, an architecture where multiple specialized LoRA adapters share a single frozen language model and communicate through tensor-level signals rather than natural language. A router network activates different cognitive specialists based on input characteristics. Across five iterations, routing-augmented responses scored up to +1.6 points higher than baseline. We identify the fundamental bottleneck: the LM head compresses rich internal representations into surface-level token probabilities, pointing toward vector-based communication as the necessary next step. Also available in French: Micro-AGI : Intelligence émergente à partir de réseaux de petits modèles de langage

Micro-AGI: Emergent Intelligence from Networks of Small Language Models

Key Points

Abstract

Cite This Study