What question did this study set out to answer?

This research aims to improve traffic signal control using a decentralized multiagent reinforcement learning approach that addresses challenges in agent interactions and model complexity.

February 26, 2026

Employing the Mean-Field Approximation Technique in Multiagent Reinforcement Learning to Control Signalized Intersections in an Urban Network

Key Points

This research aims to improve traffic signal control using a decentralized multiagent reinforcement learning approach that addresses challenges in agent interactions and model complexity.
Introduced a decentralized multiagent reinforcement learning model combining double deep Q network with mean-field approximation technique.
Utilized dual estimators in training to resolve the overestimation problem in traditional DQN.
Compared the novel approach with other algorithms to evaluate effectiveness on key performance metrics.
Experimental results show reduced waiting time and average speed improvement.
Queue length was effectively minimized compared to other tested algorithms.
The model demonstrates robustness across various traffic generation tools.

Abstract

Traffic signal control is a difficult task to ensure the performance of traffic networks in major cities around the world. Multiagent reinforcement learning (MARL) is a promising approach for traffic light management. As the number of agents increases, the learning process becomes impossible because of the curse of dimensionality and the interactions between agents. To solve this, we introduce a novel decentralized MARL-based approach combining the double deep Q network with the mean-field approximation technique (MFA). Our model eliminates the overestimation problem of the traditional DQN by using dual estimators during training. In addition, it also reduces model complexity by using the MFA to approximate the interaction within the population of agents as the interaction between a single agent and the average effect from neighboring agents. Our proposed method is compared against other algorithms to test its effectiveness. This study also provides an analysis of the influence of using different traffic generation tools (OD2Trips, DUArouter, Marouter, and DUAIterate) on the model performance. Experimental results demonstrate their effectiveness and robustness over other algorithms in terms of waiting time, average speed, and queue length.

Bookmark

Employing the Mean-Field Approximation Technique in Multiagent Reinforcement Learning to Control Signalized Intersections in an Urban Network

Key Points

Abstract

Cite This Study