We propose a framework to learn to schedule a job-shop problem (JSSP) using a graph neural network (GNN) and reinforcement learning (RL). We formulate the scheduling process of JSSP as a sequential decision-making problem with graph representation of the state to consider the structure of JSSP. In solving the formulated problem, the proposed framework employs a GNN to learn that node features that embed the spatial structure of the JSSP represented as a graph (representation learning) and derive the optimum scheduling policy that maps the embedded node features to the best scheduling action (policy learning). We employ Proximal Policy Optimization (PPO) based RL strategy to train these two modules in an end-to-end fashion. We empirically demonstrate that the GNN scheduler, due to its superb generalization capability, outperforms practically favoured dispatching rules and RL-based schedulers on various benchmark JSSP. We also confirmed that the proposed framework learns a transferable scheduling policy that can be employed to schedule a completely new JSSP (in terms of size and parameters) without further training.

Energy storage systems (ESSs) have been considered to be an effective solution to reduce the spatial and temporal imbalance between the stochastic energy generation and the demand. To effectively utilize an ESS, an approach of jointly sharing and operating an ESS has been proposed in a conceptual way. However, there is a lack of analytic approaches designed to optimize the operation of such a system considering interactions among users in a game theoretic perspective. In this study, we propose the energy capacity trading and operation (ECTO) game where each agent determines two actions, capacity trading, and the 24-hour ahead charging-discharging scheduling with the capacity that will be assigned, to minimize the energy operation cost. We then propose a distributed optimization strategy to find a generalized Nash equilibrium for the proposed ECTO game. Simulation studies show that when optimally operated, a shared ESS can decrease both the total energy operating cost and the peak-to-average ratio of the energy for the entire grid compared to the conventional ESS control strategy without ESS sharing.

Energy prediction of machine tools can deliver many advantages to a manufacturing enterprise, ranging from energy-efficient process planning to machine tool monitoring. Physics-based energy prediction models have been proposed in the past to understand the energy usage pattern of a machine tool. However, uncertainties in both the machine and the operating environment make it difficult to predict the energy consumption of the target machine reliably. Taking advantage of the opportunity to collect extensive, contextual, energy-consumption data, we discuss a data-driven approach to develop an energy prediction model of a machine tool in this paper. First, we present a methodology that can efficiently and effectively collect and process data extracted from a machine tool and its sensors. We then present a data-driven model that can be used to predict the energy consumption of the machine tool for machining a generic part. Specifically, we use Gaussian process (GP) regression, a nonparametric machine-learning technique, to develop the prediction model. The energy prediction model is then generalized over multiple process parameters and operations. Finally, we apply this generalized model with a method to assess uncertainty intervals to predict the energy consumed by any part of the machine using a Mori Seiki NVD1500 machine tool. Furthermore, the same model can be used during process planning to optimize the energy-efficiency of a machining process.

This paper describes a data-driven approach for real-time control of a physical system. Specifically, this paper focuses on the cooperative wind farm control where the objective is to maximize the total wind farm power production by using control actions as an input and measured power as an output. For real time, data-driven wind farm control, it is imperative that the optimization algorithm is able to improve a target wind farm power production by executing as small number of trial actions as possible using the wind farm power monitoring data. To achieve this goal, we develop a Bayesian ascent (BA) algorithm by incorporating into the Bayesian optimization framework a strategy that regulates the search domain, as used in the trust region method. The BA algorithm is composed of two iterative phases, namely, learning and optimization phases. In the learning phase, the BA algorithm approximates the target function using Gaussian process regression to fit the measured input and output of the target system. In the optimization phase, the BA algorithm determines the next sampling point to learn more about the target function (exploration) as well as to improve the target value (exploitation). Specifically, the sampling strategy is designed to ensure that the input is selected within a trust region to improve the target value monotonically by gradually changing the input for a target system. The results from simulation studies using an analytical wind farm power function and experimental studies using scaled wind turbines show that the BA algorithm can achieve an almost monotonic increase in the target value.