Reinforcement learning (RL) is a promising data-driven approach for adaptive traffic signal control (ATSC) in complex urban traffic networks, and deep neural networks further enhance its learning power. However, the centralized RL is infeasible for large-scale ATSC due to the extremely high dimension of the joint action space. The multi-agent RL (MARL) overcomes the scalability issue by distributing the global control to each local RL agent, but it introduces new challenges: now, the environment becomes partially observable from the viewpoint of each local agent due to limited communication among agents. Most existing studies in MARL focus on designing efficient communication and coordination among traditional Q-learning agents. This paper presents, for the first time, a fully scalable and decentralized MARL algorithm for the state-of-the-art deep RL agent, advantage actor critic (A2C), within the context of ATSC. In particular, two methods are proposed to stabilize the learning procedure, by improving the observability and reducing the learning difficulty of each local agent. The proposed multi-agent A2C is compared against independent A2C and independent Q-learning algorithms, in both a large synthetic traffic grid and a large real-world traffic network of Monaco city, under simulated peak-hour traffic dynamics. The results demonstrate its optimality, robustness, and sample efficiency over the other state-of-the-art decentralized MARL algorithms.

Exploiting reinforcement learning (RL) for traffic congestion reduction is a frontier topic in intelligent transportation research. The difficulty in this problem stems from the inability of the RL agent simultaneously monitoring multiple signal lights when taking into account complicated traffic dynamics in different regions of a traffic system. Such challenge is even more outstanding when forming control decisions on a large-scale traffic grid, where the RL action space grows exponentially with the number of intersections within the traffic grid. In this paper, we tackle such a problem by proposing a cooperative deep reinforcement learning (Coder) framework. The intuition behind Coder is to decompose the original difficult RL task as a number of subproblems with relatively easy RL goals. Accordingly, we implement Coder with multiple regional agents and a centralized global agent. Each regional agent learns its own RL policy and value functions over a small region with limited actions. Then, the centralized global agent hierarchically aggregates RL achievements from different regional agents and forms the final Q-function over the entire large-scale traffic grid. The experimental investigations demonstrate that the proposed Coder could reduce on average 30% congestions in terms of the number of waiting vehicles during high density traffic flows in simulations.

The global market for Additive Manufacturing (AM) is expected to grow, which may increase the prominence of sustainability aspects in the manufacturing process. A growing number of AM academics and practitioners have started to pay attention to the environmental and societal impacts of AM instead of only focusing on its economic aspect. Yet, AM is still not widely adopted, and the research on AM sustainability is still at the nascent stage. This paper aims to better understand AM’s sustainable adoption and seeks to address three questions: what the sustainability implications of AM are; what challenges may prevent the broad adoption of AM; and what opportunities can enable AM sustainability. The research adopts a multiple case study method to investigate six AM companies that play different roles in the AM ecosystem, including AM design, AM machine, AM material, AM service, AM education, and AM consulting. The results from these studies reveal that AM has the potential to reduce environmental and social impacts; however, it might also cause negative consequences and lead to some rebound effects. We identified 43 categories (synthesized from 199 examples) of key challenges for AM adoption and proposed 55 key solutions in moving AM towards sustainability. It is evident that AM acts as a promising digital technology for manufacturing and has the potential to pave the way for a new era of sustainable manufacturing.

Quality prediction in the continuous casting process is of great significance to the quality improvement of casting slabs. Due to the uncertainty and nonlinear relationship between the quality of continuous casting slabs (CCSs) and various factors, reliable prediction of CCS quality poses a challenge to the steel industry. However, traditional prediction models based on domain knowledge and expertise are difficult to adapt to the changes in multiple operating conditions and raw materials from various enterprises. To meet the challenge, we propose a framework with a multiscale convolutional and recurrent neural network (MCRNN) for reliable CCS quality prediction. The proposed framework outperforms conventional time series classification methods with better feature representation since the input is transformed at different scales and frequencies, which captures both long-term trends and short-term changes in time series. Moreover, we generate different category distributions based on the random undersampling (RUS) method to mitigate the impact of the skewed data distribution due to the natural imbalance of continuous casting data. The experimental results and comprehensive comparison with the state-of-the-art methods show the superiority of the proposed MCRNN framework, which has not only satisfactory prediction performance but also good potential to improve continuous casting process understanding and CCS quality.

Online map matching is essential for some location-based services, such as car navigation. However, due to GPS measurement errors and/or the lack of sufficient information, the performance of most existing online algorithms will degrade in the increasingly complex traffic environment. In this paper, we propose an adaptive online map matching algorithm called AMM. The basic idea is that AMM should be able to calibrate GPS observation data for various measurement errors and under complex urban conditions. First, we establish a collaborative evaluation model between GPS points and candidate points to effectively filter low-quality GPS measurement points, dynamically set the weights of different features and comprehensively select the best candidate points. Second, we propose a retrospective correction mechanism to correct the previous matching results when more information is available, which will help improve the accuracy of future GPS points. Furthermore, we define parameter self-tuning rules for AMM to enhance its portability by avoiding time-consuming parameter tuning steps. We conduct extensive experiments to evaluate the performance of AMM on real vehicle trajectory datasets. The experiment results show that AMM outperforms its counterparts by up to 32% in terms of accuracy and its performance in different traffic conditions is more stable.