The job shop scheduling problem (JSSP) is a well-known NP-hard combinatorial optimization problem that focuses on assigning tasks to limited resources while adhering to certain constraints. Currently, deep reinforcement learning (DRL)-based solutions are being widely used to solve the JSSP by defining the problem structure on disjunctive graphs. Some of the proposed approaches attempt to leverage the structural information of the JSSP to capture the dynamics of the environment without considering the time dependency within the JSSP. However, learning graph representations only from the structural relationship of nodes results in a weak and incomplete representation of these graphs which does not provide an expressive representation of the dynamics in the environment. In this study, unlike existing frameworks, we defined the JSSP as a dynamic graph to explicitly consider the time-varying aspect of the JSSP environment. To this end, we propose a novel DRL framework that captures both the spatial and temporal attributes of the JSSP to construct rich and complete graph representations. Our DRL framework introduces a novel attentive graph isomorphism network (Attentive-GIN)-based spatial block to learn the structural relationship and a temporal block to capture the time dependency. Additionally, we designed a gated fusion block that selectively combines the learned representations from the two blocks. We trained the model using the proximal policy optimization algorithm of reinforcement learning. Experimental results show that our trained model exhibits significant performance enhancement compared to heuristic dispatching rules and learning-based solutions for both randomly generated datasets and public benchmarks.

The job shop scheduling problem (JSSP), a renowned NP-hard combinatorial optimization problem (COP), involves sequencing finite machines for a set of jobs while adhering to defined constraints. Recent advances have seen the rise of reinforcement learning for improving JSSP solutions. This study presents an innovative end-to-end Deep Reinforcement Learning (DRL) framework for tackling the JSSP. Our DRL framework leverages convolution-augmented attention with a gate mechanism to embed the JSSP environment represented as a disjunctive graph. The proposed framework integrates convolution-augmented features for local pattern capture with multi-head self-attention (MHSA) to model global dependencies within the JSSP environment, thereby facilitating the acquisition of comprehensive graph representations. A Gate mechanism modulates feature flow between the convolution and MHSA layers, further enriching the representations. A simple multi-layer perceptron (MLP)-based action selection network generates optimal schedules sequentially. The proposed framework is trained using a proximal policy optimization (PPO) algorithm in an actor-critic framework. Experiments demonstrate the superiority of our proposed framework over existing heuristics and state-of-the-art DRL baselines, underscoring the potential of deep reinforcement learning in complex scheduling problems.