We propose a unified additive–multiplicative optimization framework, termed hybrid multiplicative gradient decomposition (HMGD), for training machine learning models. Unlike conventional gradient-based methods that rely solely on additive parameter updates, the proposed approach decomposes the gradient into complementary additive and multiplicative components, where the multiplicative term is defined through logarithmic derivative transformations to capture geometric scaling effects. This formulation enables the simultaneous modeling of linear and exponential parameter dynamics, which is particularly relevant in non-convex optimization settings and in models involving multiplicative interactions. The HMGD framework introduces separate momentum mechanisms for additive and multiplicative components, along with norm-based regularization to improve stability and promote structured sparsity in gradient updates. The method can be integrated into standard backpropagation by extending the chain rule to incorporate geometric derivatives. Empirical evaluations on multiple benchmark datasets demonstrate that HMGD achieves consistently faster convergence, improved robustness under multiplicative noise, and competitive or slightly improved performance compared to widely used optimizers such as Adam and RMSProp. Additional analysis shows that the proposed framework induces higher gradient sparsity and maintains stable optimization behavior across training. These results suggest that HMGD provides a flexible and theoretically grounded alternative for optimization in complex learning systems, particularly in scenarios involving nonlinear and multiplicative dynamics.
Sajedeh Norozpour (Wed,) studied this question.