Page 10 - Penn State Civil and Environmental Engineering Magazine

FACULTY RESEARCH
How to train your infrastructure
CEE researchers develop deep reinforcement learning framework to maintain the built environment
By Tim Schley
When an engineer chooses to inspect, repair, or replace a large, deteriorating structure, that decision could be optimized through a sequential decision-making
something well, there is a reward system, and you’re trying to find the path with the highest total reward.”
When applied to an aging civil infrastructure, Papakonstantinou explained the challenge is to identify
an asset management strategy that limits costs while maintaining long-term safety and operational use. Regularly scheduled maintenance may extend a structure’s service life, but if it occurs too often, resources are spent unnecessarily, and regular operation is disrupted. Conversely, if a needed inspection
is skipped for short-term benefits, it could lead to costly repairs and more disruptions down the road.
Every decision regarding the structure
or any of its components contributes to a long-term, system-wide strategy, but Papakonstantinou noted there is a catch: even if the timing right, inspections are not always perfect. There is a chance an
underlying problem was missed.
“If we don’t observe the conditions
of the infrastructure with certainty, how can we make optimal decisions?” Papakonstantinou said.
This uncertainty, he said, has
been difficult to incorporate into a reinforcement learning framework for real-world applications. The model needs the ability to map an infrastructure’s dynamic deterioration process while also determining in real-time the probable state of the overall system and its components.
“Let’s say we’re managing an infrastructure system with 15 interdependent components, each having 10 possible states and 10 available actions,” Papakonstantinou said. “The system would have 10 to the power of 15 potential states and actions, which is more than the number of stars and planets in our Milky Way galaxy.”
According to researchers at Penn State, intelligent system-level
infrastructure management solutions are essential for achieving resilience and sustainability in an aging and changing built environment.
framework based on artificial intelligence (AI), according to Penn State researchers.
Their new approach integrates deep reinforcement learning, a form of AI where an autonomous “agent” uses
trial and error and learns from the consequences of its decisions to discover the optimal path of actions for a given sequential decision-making scenario.
“The idea for reinforcement learning comes from the way humans learn,” said Kostas Papakonstantinou, assistant professor of civil engineering. “If you do
10 CEE NEWSLETTER • VOLUME 36, 2020