Enhanced Random Ensemble Mixture: Weight Referring and Merging

Reinforcement learning (RL) is widely used to learn sequential decision-making policies in complex environments. Deep Q-network (DQN) extends Q-Learning using deep neural networks, enabling learning in high-dimensional state spaces. However, conventional DQN-based approaches can exhibit variability in learning stability and convergence speed even under similar training conditions. Random Ensemble Mixture (REM) has been introduced to improve stability by combining multiple Q-value estimates, but it typically requires running multiple models simultaneously, which increases computational cost. This paper proposes an enhanced DQN method that integrates REM with a Weight Referring and Merging (WRM) mechanism to improve training stability and efficiency. The proposed approach updates a single primary agent using standard DQN learning while maintaining diversity among auxiliary agents by selectively referring to and partially merging weights from the primary network. Q-values from the primary and auxiliary agents are then combined through REM to produce the final value estimate for action selection. Experiments in the Catch Game environment indicate that the proposed method reaches stable performance earlier than a baseline DQN and reduces training time under the tested configuration (approximately 78%). While the results are encouraging in this environment, further evaluation on additional benchmarks is required to assess broader applicability.