PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
May 31, 20260 citationsOpen Access

Per-Channel Quantile Fullgrid Probing for Small Language Model Observation

View Full Paper
HCHo Yiing Chen

Key Points

  • This research aims to develop a probing protocol that maintains high observation coverage for language models within a fixed computational budget.
  • Dense activation probing protocol targeting small to mid-sized language models.
  • Utilized a per-channel quantile binning approach with a calibration probe corpus of 1200 samples.
  • Conducted thirteen runs across seven model families ranging from 124M to 7.6B parameters.
  • Achieved at least 90.6% coverage in every run, with a median of 94.95% and a maximum of 97.05%.
  • Higher coverage noted with per-channel binning and a larger probe budget compared to prior configurations.
  • Recommended next step includes a controlled 2x2 ablation study.

Abstract

We describe a dense activation probing protocol for small to mid-sized language models that operates within a fixed cell budget while maintaining high observation coverage. The protocol combines per-channel quantile binning, an expanded calibration probe corpus (P=1200), and an additive shard merge step that allows heterogeneous compute resources to contribute. Across thirteen runs spanning seven model families and parameter counts from 124M to 7.6B, the protocol reaches at least 90.6 percent coverage on every run, with a median of 94.95 percent and a maximum of 97.05 percent. Per-channel binning combined with a larger probe budget was associated with higher coverage compared to earlier pooled-bin, smaller-corpus configurations on the same models, but the isolated contribution of each factor remains untested in this draft. A controlled 2x2 ablation is the recommended next experiment. The pipeline is organized for reproducibility and can be released alongside the run artifacts. It runs on commodity hardware.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Ho Yiing Chen (2026) studied this question.

synapsesocial.com/papers/6a1bd1f65783ba022b6fd609https://doi.org/10.5281/zenodo.20448674
Ask AI
Helpful
Bookmark
Share
View Full Paper