Page 16 - Penn State Civil and Environmental Engineering Magazine

FACULTY RESEARCH
Using machine learning to improve traffic congestion
By Jessica Hallman
Associate Professor Vikash Gayah and researchers in Penn State’s College of Information Sciences and
Technology are advancing work that utilizes machine learning methods to improve traffic signal control at urban intersections around the world.
“What makes our work in this area unique is that we are tackling the problem at multiple spatial scales,” Gayah said. “We started at the individual intersection, but we have now moved to network-wide control using these advanced techniques, and our initial work is showing excellent promise.”
Through their work, the researchers explored the use of reinforcement learning—training algorithms
to learn how to achieve a
complex goal through a reward system—to examine patterns and communication of traffic signal control.
The researchers studied real-world traffic data from major cities in
the United States and China to analyze intersection congestion. Then, through a series of studies, they developed an in-depth reinforcement learning method that could positively impact traffic flow by learning the best strategies for a traffic flow pattern.
Zhenhui “Jessie” Li, associate professor of information sciences and technology, explained that reinforcement learning is, for example, similar to using positive reinforcement to improve a child’s behavior.
“When teaching kids [using positive reinforcement], if they do a good job you give them some reward,”
Li said. “If they do a bad job,
then you just don’t give them the reward.”
She continued, “This is the same idea. For a traffic signal, if it does
a bad job, cars can get congested and the reward will go down. Then if it’s doing a good job because the cars are saving time at intersections, it will get some reward. For our setting, the reward would be that the cars take a shorter time to get to their destination. But that is something that is hard to directly quantify because travel time is the result of a long sequence of traffic signals, and it is also affected by other factors such as free moving speed.”
The major contribution of the latest paper is to scale the traffic signal control into more intersections, according to Chacha Chen, doctoral student in the College of IST and lead author on one of the papers.
“Previously, we’ve typically done experiments on at most 16 intersections,” Chen said. “So the method is not new.”
Li added, “It’s the first time we can scale to one thousand intersections, when it used to be less than 20.
It’s 50 times more than [it was] previously, making it truly city-wide.”
The researchers are hopeful that their work could help cities around the world optimize how they use intersections as resources.
“As urbanization continues, and more people move into a city, if
you don’t maximize the use of
the resource of a road, you would probably end up building more roads to satisfy the needs of the
city for the traffic,” Li said. “But the land could be used to build a school or a hospital.
“So the motivation behind our work is to explore if we can use data as a sort of a new resource to help us optimize the use of other resources,” she added.
Collaborators on the work include Guanjie Zheng and Hua Wei, doctoral students in the Penn State College of IST; Huichu Zhang, Xinshi Zang, Weinan Zhang and Yanmin Zhu of Shanghai Jiao Tong University; Jie Feng and Yong Li of Tsinghua University; Yuanhao Xiong of Zhejiang University; and Kai Xu of Shanghai Tianrang Intelligent Technology Co. Ltd.
16 CEE NEWSLETTER • VOLUME 36, 2020