A Distributed Multi-Agent Reinforcement Learning With Graph Decomposition Approach for Large-Scale Adaptive Traffic Signal Control

Shan Jiang, Yufei Huang, Mohsen Jafari, Mohammad Jalayer

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

With the emerging connected-vehicle technologies and smart roadways, the need for intelligent adaptive traffic signal controls (ATSC) is more than ever before. This paper first proposes an Accumulated Exponentially Weighted Waiting Time-based Adaptive Traffic Signal Control (AEWWT-ATSC) model to calculate priorities of roadways for signal scheduling. As the size of the traffic network grows, it adds great complexities and challenges to computational efficiencies. Considering this, we propose a novel Distributed Multi-agent Reinforcement Learning (DMARL) with a graph decomposition approach for large-scale ATSC problems. The decomposition clusters intersections by the level of connectivity (LoC), defined by the average residual capacities (ARC) between connected intersections, enabling us to train subgraphs instead of the entire network in a synchronized way. The problem is formulated as a Markov Decision Process (MDP), and the Double Dueling Deep Q Network with Prioritized Experience Replay is utilized to solve it. Under the optimal policy, the agents can select the optimal signal durations to minimize the waiting time and queue size. In evaluation, we show the superiority of the AEWWT-ATSC based RL methods in different densities and demonstrate the DMARL with a graph decomposition approach on a large graph in Manhattan, NYC. The approach is generic and can be extended to various types of use cases.

Original languageEnglish (US)
Pages (from-to)14689-14701
Number of pages13
JournalIEEE Transactions on Intelligent Transportation Systems
Volume23
Issue number9
DOIs
StatePublished - Sep 1 2022

All Science Journal Classification (ASJC) codes

  • Automotive Engineering
  • Mechanical Engineering
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'A Distributed Multi-Agent Reinforcement Learning With Graph Decomposition Approach for Large-Scale Adaptive Traffic Signal Control'. Together they form a unique fingerprint.

Cite this