The smart factory is at the heart of Industry 4.0 and is the new paradigm for establishing advanced manufacturing systems and realizing modern manufacturing objectives such as mass customization, automation, efficiency, and self-organization all at once. Such manufacturing systems, however, are characterized by dynamic and complex environments where a large number of decisions should be made for smart components such as production machines and the material handling system in a real-time and optimal manner. AI offers key intelligent control approaches in order to realize efficiency, agility, and automation all at once. One of the most challenging problems faced in this regard is uncertainty, meaning that due to the dynamic nature of the smart manufacturing environments, sudden seen or unseen events occur that should be handled in real-time. Due to the complexity and high-dimensionality of smart factories, it is not possible to predict all the possible events or prepare appropriate scenarios to respond. Reinforcement learning is an AI technique that provides the intelligent control processes needed to deal with such uncertainties. Due to the distributed nature of smart factories and the presence of multiple decision-making components, multi-agent reinforcement learning (MARL) should be incorporated instead of single-agent reinforcement learning (SARL), which, due to the complexities involved in the development process, has attracted less attention. In this research, we will review the literature on the applications of MARL to tasks within a smart factory and then demonstrate a mapping connecting smart factory attributes to the equivalent MARL features, based on which we suggest MARL to be one of the most effective approaches for implementing the control mechanism for smart factories.

Background: Today’s production facilities must be efficient in both manufacturing and maintenance. Efficiency enables the company to maintain the required output while reducing production effort or costs. With the increasing interest in process automation and the Internet of things since Industry 4.0 was introduced, such shop floors are growing in complexity. Every component of the production needs to be continuously monitored, which is the basis for predictive maintenance (PdM). To predict when maintenance is needed, the components’ conditions are monitored with the help of a condition monitoring (CM) system. However, this task is difficult for human employees, as the monitoring and analysis is very demanding. To overcome this, machine learning (ML) can be applied to ensure more efficient production. Methods: This paper aims to investigate the application of ML techniques for CM and PdM in the manufacturing sector. For this reason, a systematic mapping study (SMS) is conducted in order to structure and classify the current state of research and identify potential gaps for future investigation. Relevant literature was considered between January 2011 and May 2021. Results: Based on the guidelines for SMSs and previously defined research questions, existing publications are examined and a systematic overview of the current state of the research domain is provided. Conclusions: Techniques such as reinforcement learning and transfer learning are underrepresented, but increasingly attracting more attention. The findings of this study suggest that the most promising results belong to the applications of hybrid ML methods, where a set of methods are combined to build a more powerful model.

The semiconductor industry is strongly increasing the production capacities and the product portfolio for a wide range of applications that are needed in the worldwide supply chains e.g. the automotive, computer and security industry. The complex manufacturing processes require more automation, dig- italisation and IoT frameworks, especially for highly automated semiconductor manufacturing plants. Over the last years, this industry spent much effort to control highly sensitive materials in production by product monitoring using advanced process control by various sensors in production. Nevertheless, until today, sensor integration, especially for such sensors that are not supported by the equipment vendors, is time-consuming and complicated. This article aims to use a micro-service-based approach by Eclipse Arrowhead as an open-source microservice architecture and implementation platform [1]. This architecture is an easy and powerful framework that can be used for multiple sensor applications to control the manufacturing material flow in a modern semiconductor plant with a high product mix. The article describes how the engineering process was designed, the architecture of the use case and the main benefits in the operational business are shown.