Affiliations: [a] Intelligent Transportation System Research Center, Southeast University, Nanjing, China
| [b] School of Mathematics, and Research Center for Complex Systems and Network Sciences, Southeast University, Nanjing, China
| [c] School of Automation, Southeast University, Nanjing, China
Correspondence:
[*]
Corresponding author. Jinde Cao, School of Mathematics, and Research Center for Complex Systems and Network Sciences, Southeast University, 2 Dongnandaxue Rd, Nanjing, China. E-mail: [email protected].
Abstract: Aiming at the problem of mainline congestion and ramp queue spill in urban expressway, an optimized on-ramp control method based on reinforcement learning is put forward. Online reinforcement learning algorithm is used to optimize on-ramp control regulation by taking the metering rate as the action, the length of ramp queue, the throughput and the occupancy rate of the interweaving area as the state, and the volume of the road network as the reward function. By iterating the value function with actual behavior, the proposed method can avoid the establishment of an accurate traffic model and the reliance on prior knowledge. Meanwhile, the real-time update of the value function Q compensates for the defect of control hysteresis. Compared with the classical method in simulation scenarios of Nanjing Kazimen Expressway, the average delay of the proposed method is reduced by 16.83%, the total delay reduced by 15.83%, the average speed enhanced by 6.80%, and the total travel time decreased by 5.22%; the average queue length of the on-ramp decreased by 89%; the average occupancy rate of the weaving area is decreased by 2.42% at rush hours, and the average traffic volume increased by 109veh/h.
