We consider a train scheduling problem in which both local and express trains are to be scheduled. In this type of train scheduling problem, the key decision is determining the overtaking stations at which express trains overtake their preceding local trains. This problem has been successfully modeled via mixed integer programming (MIP) models. One of the obvious limitation of MIP-based approaches is the lack of freedom to the choices objective and constraint functions. In this paper, as an alternative, we propose an approach based on reinforcement learning. We first decompose the problem into subproblems in which a single express train and its preceding local trains are considered. We, then, formulate the subproblem as a Markov decision process (MDP). Instead of solving each instance of MDP, we train a deep neural network, called deep Q-network (DQN), which approximates Q-value function of any instances of MDP. The learned DQN can be used to make decision by choosing the action which corresponds to the maximum Q-value. The advantage of the proposed method is the ability to incorporate any complex objective and/or constraint functions. We demonstrate the performance of the proposed method by numerical experiments.

The disturbance of local areas with complex railway networks and high traffic density not only impedes the efficient use of rail networks in those areas, but also propagates delays to the entire railway network. This has motivated research on train rescheduling problems in high-density local areas to minimize train delays by modifying their planned arrival and departure times. In this paper, we present a train rescheduling method based on Q-learning in reinforcement learning. More specifically, we used deep neural networks to approximate the action-value function, and the underlying Markov decision process (MDP) is based on the alternative graph formulation for the train rescheduling problem. In the proposed MDP formulation, the status of the alternative graph corresponding to the current schedule is defined as the state, and the alternative arc corresponds to the action the agent can take. The MDP is approximately solved via deep Q-learning in which deep neural networks are used to approximate the action-value function in Q-learning. Although the size of the alternative graph depends on the number of trains, our MDP formulation is independent of the number of trains, which makes the proposed method more scalable. The evaluation of the method was performed on a simple railway network and a real-world example in Seoul, South Korea, with randomly generated initial train schedules and train delays. The experimental result showed that the proposed method is comparable to the mixed-integer-linear-programming (MILP)-based exact approach with respect to the quality of the solution.

Previous studies have proposed various frameworks and algorithms to optimize routes to reduce total transportation cost, which accounts for over 29.4% of overall logistics costs. However, it is very hard to find cases in which mathematical models or algorithms are applied to practical business environment cases which require reusable packaging, especially daily operating logistics services like convenience store support systems. Most previous studies have considered developing an optimal algorithm which can solve the mathematical problem within a practical amount of time while satisfying all constraints, such as the capacity of delivery and pick-up, and hard or soft time windows. For daily delivery and pick-up services, like those supporting several convenience stores, it is required to consider the unit transporting the container, as well as the demand, capacity of trucks, travel distance, and traffic congestion. In particular, reusable transport containers and trays should be regarded as important assets of logistics centers. However, if the mathematical model focuses on only satisfying constraints related to delivery and not considering the cost of trays, it is often to leave the empty trays on the pick-up points when there is not enough space in the track. In this study, we propose a mathematical model for optimizing delivery and pick-up plans by extending the general vehicle routing problem of simultaneous delivery and pick-up with time windows, while considering left-over cost. With numerical experiments it has been proved that the proposed model may reduce the total delivery cost. Also, it seems possible to apply the proposed approach to the various logistics businesses which require reusable transport containers like shipping containers, refrigerating containers, trays, and pallets.