Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
April 14, 2026Open Access

Operational Experience as a Performance Multiplier in AI Assistants: A Controlled Study with Triple-Judge Blind Evaluation

View Full Paper
Ask AI
Bookmark
Share

Authors

NRNuraliev RavshanRARychkova Anastasia

Discussion

Loading...

Member takes

Overview

Controlled experiment shows that accumulated experience improves AI assistant performance on specific tasks, suggesting new improvement strategies.

Key Points

  • This research aims to measure the impact of accumulated operational experience on the performance of AI assistants in specific tasks.
  • Conducted a controlled experiment using a 2×2 factorial design with 8 conditions.
  • Utilized 50 real-world questions from public sources for evaluation.
  • Received 1,200 blind judgments from three independent AI judges (GPT-5.4, Gemini 3.1 Pro, Claude Opus 4.6).
  • Experience-augmented AI assistant significantly outperformed baseline models (Cohen's d = 1.07, p < 10⁻²⁵).
  • Performance increase was domain-specific with a +1.65 standard deviation on operational tasks.
  • Independent classification revealed 2.2× more genuinely experiential content in the experience-augmented model.

Cite This Study

Ravshan et al. (2026) studied this question.

synapsesocial.com/papers/69ddd938e195c95cdefd6967https://doi.org/10.5281/zenodo.19533311
View Full Paper
Ask AI
Bookmark
Share