The objective of this paper is to examine the use and applications of reinforcement learning (RL) techniques in the production planning and control (PPC) field addressing the following PPC areas: facility resource planning, capacity planning, purchase and supply management, production scheduling and inventory management. The main RL characteristics, such as method, context, states, actions, reward and highlights, were analysed. The considered number of agents, applications and RL software tools, specifically, programming language, platforms, application programming interfaces and RL frameworks, among others, were identified, and 181 articles were sreviewed. The results showed that RL was applied mainly to production scheduling problems, followed by purchase and supply management. The most revised RL algorithms were model-free and single-agent and were applied to simplified PPC environments. Nevertheless, their results seem to be promising compared to traditional mathematical programming and heuristics/metaheuristics solution methods, and even more so when they incorporate uncertainty or non-linear properties. Finally, RL value-based approaches are the most widely used, specifically Q-learning and its variants and for deep RL, deep Q-networks. In recent years however, the most widely used approach has been the actor-critic method, such as the advantage actor critic, proximal policy optimisation, deep deterministic policy gradient and trust region policy optimisation.

This study presents a review of optimization models for tactical production planning. The objective of this research is to identify streams and future research directions in this field based on the different classification criteria proposed. The major findings indicate that: (1) the most popular production-planning area is master production scheduling with a big-bucket time-type period; (2) most of the considered limited resources correspond to productive resources and, to a lesser extent, to inventory capacities; (3) the consideration of backlogs, set-up times, parallel machines, overtime capacities and network-type multisite configuration stand out in terms of extensions; (4) the most widely used modelling approach is linear/integer/mixed integer linear programming solved with exact algorithms, such as branch-and-bound, in commercial MIP solvers; (5) CPLEX, C and its variants and Lindo/Lingo are the most popular development tools among solvers, programming languages and modelling languages, respectively; (6) most works perform numerical experiments with random created instances, while a small number of works were validated by real-world data from industrial firms, of which the most popular are sawmills, wood and furniture, automobile and semiconductors and electronic devices.