PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
April 16, 20260 citationsOpen Access

Orthogonal Probing and the Geometry of LLM Entropy: Empirical Evidence for Axis-Rotation Auditing of LLM-Generated Artifacts

View Full Paper
MBMartin Brodeur

Key Points

  • This research investigates the effectiveness of orthogonal auditing methods in enhancing LLM artifact discovery and vulnerability assessment.
  • Implemented Orthogonal Auditing by Rotation (OAR) using cosine distance in embedding space.
  • Conducted single-pass T2 experiments on eight external codebases to compare OAR with baseline methods.
  • Performed controlled experiments on diverse security prompts across various topics and codebases.
  • Analyzed defect density predictions using session-touch count.
  • Achieved a 1.5--2.5x increase in discovery compared to same-axis audits.
  • Confirmed a 6/8 win rate over topic-diverse prompts with measurable outcomes.
  • Demonstrated that no single axis accounted for more than 30% of significant findings across codebases.
  • Identified 76 private security advisories across eight open-source projects, with critical findings inaccessible by traditional methods.

Abstract

When LLMs audit artifacts from the same semantic direction used to generate them, they re-enter the same compressed manifold region, producing rising confidence while discovery stalls --- a failure mode we term Generator-Auditor Symmetry (GAS). Orthogonal Auditing by Rotation (OAR) escapes this by selecting probe directions via cosine distance in embedding space, accumulating confirmed defect classes in a persistent registry, and stopping when the false-positive rate signals entropy exhaustion (P2 floor). Single-pass T2 experiments on eight external codebases (vLLM, LangChain, Gradio, MLflow, Superset, LiteLLM, Dify, Open WebUI) yield a 1.5--2.5x exclusive discovery advantage over same-axis baselines, with 85--100% non-overlap between rotation and repetition findings. A resource-equivalent controlled experiment (T13-multi) comparing OAR against 4 genuinely diverse-topic security prompts on the same 8 codebases yields a 6/8 win rate (mean Delta E = +1.125, mean Jaccard = 0.187), confirming that axis rotation provides manifold coverage distinct from topic diversity. Full-campaign rotation on a 350K-line production codebase (156+ sessions) produces a cumulative 3--5x yield advantage as the saturation curve compounds across axes. No single axis captures >30% of critical findings in any codebase. The method generalizes across Python, TypeScript, Java, and nine structurally disjoint application domains. Beyond empirical yield, OAR provides an epistemically grounded coverage framework for LLM-based auditing: each rotation makes a falsifiable behavioral claim about reducing the unseen vulnerability surface, a property no topic-diversity or prompt-variation strategy can replicate. (The claim is behavioral and operational --- grounded in the observed 85--100% non-overlap between rotation and repetition findings; it does not depend on confirmed activation-space measurement, which is the target of T4.) A vocabulary-matched controlled experiment (T1, 6/6 cells, 3 models) provides directionally consistent evidence that axis direction, not vocabulary specificity, drives the effect (pilot result; no per-cell p-values or power analysis; formal replication at scale is T1-ext). Persistent homology (T6) confirms the underlying manifold topology is generic to LLM semantic organization, not defect-specific --- predicting domain-general applicability. Session-touch count predicts defect density (r ≈ 0.71, p = 9.0), all from axis directions the same-axis baseline structurally cannot reach. The theoretical apparatus underlying these results --- 92 conjectures on LLM manifold geometry, the formal consequences of GAS and CCD, and the epistemological foundations of oracle-free auditing --- is developed in the companion paper.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Martin Brodeur (2026) studied this question.

synapsesocial.com/papers/69e07d8f2f7e8953b7cbe756https://doi.org/10.5281/zenodo.19562109
Ask AI
Helpful
Bookmark
Share
View Full Paper