Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
May 10, 2026Open Access

SurpriseOpt: An Adaptive First-Order Optimizer Driven by Boredom

View Full Paper
Ask AI
Bookmark
Share

Authors

USUwe Stöhr

Discussion

Loading...

Member takes

Overview

Randomized trial shows faster convergence of SurpriseOpt compared to Adam, suggesting improved optimization.

Key Points

  • The aim is to introduce SurpriseOpt, an optimizer that uses adaptive techniques to enhance convergence speed.
  • Developed SurpriseOpt with adaptive gating functions to manage inertia based on gradient and second-moment ratios.
  • Implemented boredom mechanism to adjust learning rates when low surprises are detected.
  • Compared performance against Adam across various optimization tasks.
  • SurpriseOpt converged significantly faster than Adam in various tests, demonstrating improved efficiency.
  • The mechanism effectively modulated the learning rate based on the detected boredom state.
  • The approach showed enhanced performance on multiple optimization benchmarks.

Cite This Study

Uwe Stöhr (2026) studied this question.

synapsesocial.com/papers/6a0021fec8f74e3340f9d05ehttps://doi.org/10.5281/zenodo.20081833
View Full Paper
Ask AI
Bookmark
Share