Traffic flow forecasting is an important problem for the successful deployment of intelligent transportation systems, which has been studied for more than two decades. In recent years, deep learning methods are emerging to serve as the benchmark tool for traffic flow forecasting due to its superior prediction performance. However, most studies are based on simple deep learning methods that can not capture inter- and intra-day traffic patterns as well as the correlation between contextual factors like the weather and the traffic flow. In this paper, we propose a novel deep-learning-based method for daily traffic flow forecasting where incorporating contextual factors and traffic flow patterns can be critical. First, a particular convolutional neural network (CNN) is deployed to extract inter- and intra- day traffic flow patterns. Then extracted features are fed into long short-term memory (LSTM) units to learn the intra-day temporal evolution of traffic flow. Finally, contextual information of historical days is integrated to enhance the prediction performance. Through a real-data case study, we show that the proposed approach achieves over 90% prediction accuracy which greatly outperforms existing benchmark methods and its forecasting performance is robust under various scenarios.

Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Traffic signal control is a critical method that ensures the efficiency of traffic flow in cities across the world. There are massive studies that focus on generating optimal signal timing plans. Most of the these studies are model-based, where the signal plan is determined by optimization models with fixed parameters. Reinforcement learning (RL) is a model-free method that learns the optimal control policy over time which avoids limitations of traditional methods. In the past a few years, deep RL methods become particularly attractive since deep neural networks can provide scalable learning capability given the complex traffic condition in real life. In this manuscript, we develop a deep actor-critic method that can provide efficient traffic signal plans. First, a novel deep neural network model is applied to mine the recent traffic condition information as a sequence of temporally sequential image representations of the intersection, which improves from relying solely on limited traffic information such as queue length; then the deep neural network model is integrated with an actor-critic model which avoids drawbacks from the value-based or the policy-based methods. Simulation experiments illustrate that the deep actor-critic method outperforms the classic model-based method and several existing deep reinforcement learning methods according to different measures such as queue length, average delay time, and throughput.

The purpose of this paper is to design a route and speed optimization method to simultaneously reduce sailing cost and time, considering Emission Control Areas (ECAs) regulations and weather conditions. To solve it, the Non-dominated Sorting Genetic Algorithm (NSGA-II) was used to find the Pareto-optimal set, and the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) was used to find the trade-off solution from the Pareto-optimal set. The proposed model was then applied to a service route around the US Coast, based on different fuel prices in ECAs. The results show that the proposed method can reduce the total sailing cost and time compared with other methods, and can help shipping companies effectively cope with fuel price fluctuations. Moreover, we also determined the best time interval for changing ship route and speed in terms of minimizing the sailing cost.