We consider a train scheduling problem in which both local and express trains are to be scheduled. In this type of train scheduling problem, the key decision is determining the overtaking stations at which express trains overtake their preceding local trains. This problem has been successfully modeled via mixed integer programming (MIP) models. One of the obvious limitation of MIP-based approaches is the lack of freedom to the choices objective and constraint functions. In this paper, as an alternative, we propose an approach based on reinforcement learning. We first decompose the problem into subproblems in which a single express train and its preceding local trains are considered. We, then, formulate the subproblem as a Markov decision process (MDP). Instead of solving each instance of MDP, we train a deep neural network, called deep Q-network (DQN), which approximates Q-value function of any instances of MDP. The learned DQN can be used to make decision by choosing the action which corresponds to the maximum Q-value. The advantage of the proposed method is the ability to incorporate any complex objective and/or constraint functions. We demonstrate the performance of the proposed method by numerical experiments.

Previous studies have proposed various frameworks and algorithms to optimize routes to reduce total transportation cost, which accounts for over 29.4% of overall logistics costs. However, it is very hard to find cases in which mathematical models or algorithms are applied to practical business environment cases which require reusable packaging, especially daily operating logistics services like convenience store support systems. Most previous studies have considered developing an optimal algorithm which can solve the mathematical problem within a practical amount of time while satisfying all constraints, such as the capacity of delivery and pick-up, and hard or soft time windows. For daily delivery and pick-up services, like those supporting several convenience stores, it is required to consider the unit transporting the container, as well as the demand, capacity of trucks, travel distance, and traffic congestion. In particular, reusable transport containers and trays should be regarded as important assets of logistics centers. However, if the mathematical model focuses on only satisfying constraints related to delivery and not considering the cost of trays, it is often to leave the empty trays on the pick-up points when there is not enough space in the track. In this study, we propose a mathematical model for optimizing delivery and pick-up plans by extending the general vehicle routing problem of simultaneous delivery and pick-up with time windows, while considering left-over cost. With numerical experiments it has been proved that the proposed model may reduce the total delivery cost. Also, it seems possible to apply the proposed approach to the various logistics businesses which require reusable transport containers like shipping containers, refrigerating containers, trays, and pallets.