# RL_signals **Repository Path**: greghan/RL_signals ## Basic Information - **Project Name**: RL_signals - **Description**: No description available - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2020-11-13 - **Last Updated**: 2020-12-19 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Reinforcement Learning for Traffic Signal Control The aim of this repository is to offering comprehensive **dataset**, **simulator**, relevant **papers** and **survey** to anyone who may wish to start investigation or evaluate a new algorithm. ## Table of contents - [Key paper list](#key-paper-list) - [Open Datasets](#open-datasets) - [Traffic Simulator](#traffic-simulator) - [A comprehensive survey](#survey) ## Key paper list | Method | Paper | Published | Notes | Code | Demo video | Poster| | :------------- | :------------- | :-------- | :-----: | :-----: | :-----: | :-----: | | MPLight | [Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control](https://chacha-chen.github.io/files/chacha-AAAI2020.pdf) | AAAI'2020 | A combination of PressLight and FRAP | - |[Demo](https://traffic-signal-control.github.io/a-thousand-lights.html) | -| |CoLight |[CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://sites.psu.edu/huawei/2019/09/15/colight-cikm-2019/) | CIKM'19 | Attention-based coordination| [Code](https://github.com/wingsweihua/colight) | N/A | [poster](https://github.com/traffic-signal-control/RL_signals/blob/master/posters/CIKM19-colight-poster.pdf)| |PressLight|[PressLight: Learning Max Pressure Control to Coordinate Traffic Signals in Arterial Network](https://sites.psu.edu/huawei/2019/06/17/presslight-kdd-2019/)|KDD'19| Pressure-based coordination| [Code](https://github.com/wingsweihua/presslight) |[Demo](https://www.kdd.org/kdd2019/accepted-papers/view/presslight-learning-max-pressure-control-for-signalized-intersections-in-ar) | [poster](https://github.com/traffic-signal-control/RL_signals/blob/master/posters/KDD19-presslight-poster.pdf) | | FRAP | [Learning Phase Competition for Traffic Signal Control](http://www.personal.psu.edu/~gjz5038/paper/cikm2019_frap/cikm2019_frap_paper.pdf) | CIKM'19 | Our most powerful single intersectiton control model | [Code](https://github.com/gjzheng93/frap-pub) | N/A |[poster](https://github.com/traffic-signal-control/RL_signals/blob/master/posters/cikm2019_frap.pdf)| | MetaLight | MetaLight: Value-based Meta-reinforcement Learning for Traffic Signal Control |AAAI'2020 | Meta-RL for traffic signal control | [Code](https://github.com/zxsRambo/metalight) |-|-| |DemoLight|[Learning Traffic Signal Control from Demonstrations](https://dl.acm.org/citation.cfm?id=3357384.3358079) |CIKM'19 | Learn from expert demonstrations | [Code](https://github.com/xyh97/DemoLight) |N/A| [poster](https://github.com/traffic-signal-control/RL_signals/blob/master/posters/cikm-demolight.pdf)| | IntelliLight|[IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control](https://www.kdd.org/kdd2018/accepted-papers/view/intellilight-a-reinforcement-learning-approach-for-intelligent-traffic-ligh) | KDD'18|First try on RL signal control. The base of all the methods| N/A | [Demo](https://www.kdd.org/kdd2018/accepted-papers/view/intellilight-a-reinforcement-learning-approach-for-intelligent-traffic-ligh)|[poster](https://github.com/traffic-signal-control/RL_signals/blob/master/posters/KDD18-intelliLight.pdf) | | CityFlow |[CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://arxiv.org/abs/1905.05217) | WWW'19 Demo| Simulator | [Code](https://github.com/cityflow-project/CityFlow)| [Demo](https://cityflow-project.github.io/) | N/A | ## Open datasets We provide different traffic datasets, each includes both road network (roadnet.json) and traffic flow file (flow.json), whose formats are defined in [Roadnet File Format](https://cityflow.readthedocs.io/en/latest/roadnet.html) and [Flow File Format](https://cityflow.readthedocs.io/en/latest/flow.html) respectively.
*All methods are measured in Average Travel Time (in seconds) under CityFlow simulator.
# Dataset name Number of Intersections Time Span (Seconds) Description Referred result* Referred method
1 hangzhou_1x1_bc-tyc_18041607_1h 1 3600 These datasets are based on camera data in Hangzhou. Due to the lack of records about turning vehicles, the turning ratios of each dataset are fixed, with 10% as turning left, 60% as going straight, and 30% as turning right. The turning-right vehicles are discarded since they are not under the control of traffic lights. There are one left-turn lane and one straight lane in each direction in each roadnet. 221.03 SOTL
2 hangzhou_1x1_bc-tyc_18041608_1h 1 3600 334.72 SOTL
3 hangzhou_1x1_bc-tyc_18041610_1h 1 3600 213.20 SOTL
4 hangzhou_1x1_kn-hz_18041607_1h 1 3600 72.48 SOTL
5 hangzhou_1x1_kn-hz_18041608_1h 1 3600 64.10 SOTL
6 hangzhou_1x1_qc-yn_18041607_1h 1 3600 117.24 SOTL
7 hangzhou_1x1_qc-yn_18041608_1h 1 3600 131.99 SOTL
8 hangzhou_1x1_sb-sx_18041607_1h 1 3600 173.85 SOTL
9 hangzhou_1x1_sb-sx_18041608_1h 1 3600 290.00 SOTL
10 hangzhou_1x1_tms-xy_18041607_1h 1 3600 214.77 SOTL
11 hangzhou_1x1_tms-xy_18041608_1h 1 3600 325.32 SOTL
12 syn_1x1_uniform_200_1h 1 3600 These datasets are generated artificially. The vehicles enter the road network uniformly with a fixed entering ratio chosen from 200, 400 and 600 vehicles per hour. 61.44 SOTL
13 syn_1x1_uniform_400_1h 1 3600 133.40 SOTL
14 syn_1x1_uniform_600_1h 1 3600 189.11 SOTL
15 hangzhou_4x4_gudang_18010207_1h 16 3600 The road network contains 16 intersections in a 4x4 grid. Each intersection has four incoming approaches and four outgping approaches, and each approach has three lanes (left-turn, through and right-turn respectively). The traffic flow data is based on camera data in Hangzhou. Necessary simplification is done due to the low quality of the real-world data. • Traffic volume: the traffic volume is derived from camera data at Hangzhou. • Turning ratio: 10% (turning left), 60%(going straight) and 30% (turning right). This is synthesized from the statistics of taxi GPS data. 240.97 MaxPressure
16 syn_1x3_gaussian_500_1h 3 3600 The road network contains 16 intersections in a 4x4 grid. Each intersection has four incoming approaches and four outgping approaches, and each approach has three lanes (left-turn, through and right-turn respectively). • Traffic volume: All the vehicles enter and leave the network from the rim edges.For each entering edge, the number of the vehicles generated is sampled from a Gaussian distribution with mean as 500 vehicles/hour/lane. • Turning ratio: 10% (turning left), 60%(going straight) and 30% (turning right) 422.95 MaxPressure
17 syn_2x2_gaussian_500_1h 4 3600 477.71 MaxPressure
18 syn_3x3_gaussian_500_1h 9 3600 631.75 MaxPressure
19 syn_4x4_gaussian_500_1h 16 3600 689.68 MaxPressure
20 Manhattan 2510 3600
## Survey [A Survey on traffic signal control](https://arxiv.org/abs/1904.08117) ## Team - [Zhenhui Jessie Li](https://faculty.ist.psu.edu/jessieli/Site/index.html) (Associate professor, Penn State) - [Hua Wei](http://personal.psu.edu/hzw77/index.html) (PhD, Penn State University) - [Guanjie Zheng](http://www.personal.psu.edu/~gjz5038/) (PhD, Penn State University) - [Chacha Chen](https://chacha-chen.github.io/) (PhD, Penn State University) - [Nan Xu](https://sites.google.com/site/xunannancy/home) (PhD, University of Southern California) - [Yuanhao Xiong](https://xyh97.github.io/) (PhD, University of Los Angelos) - Kan Wu (PhD, Penn State University) - Xinshi Zang (Bachelor, Shanghai Jiao Tong University) - Huichu Zhang (PhD, Shanghai Jiao Tong University) - Jie Feng (PhD, Tsinghua University)