In today’s volatile supply chain (SC) environment, competition has shifted beyond individual companies to the entire SC ecosystem. Reducing overall SC costs is crucial for success and benefits all participants. One effective approach to achieve this is through digital transformation, enhancing SC coordination via information sharing, and establishing decision policies among entities. However, the risk of unauthorized leakage of sensitive information poses a significant challenge. We aim to propose a Privacy-preserving Multi-agent Reinforcement Learning (PMaRL) method to enhance SC visibility, coordination, and performance during inventory management while effectively mitigating the risk of information leakage by leveraging machine learning techniques. The SC inventory policies are optimized using multi-agent reinforcement learning with additional SC connectivity information to improve training performance. The simulation-based evaluation results illustrate that the PMaRL method surpasses traditional optimization methods in achieving cost performance comparable to full visibility methods, all while preserving privacy. This research addresses the dual objectives of information security and cost reduction in SC inventory management, aligning with the broader trend of digital transformation.

In recent years, model-free multi-agent reinforcement learning (MaRL) has become a powerful tool for learning effective policies to solve optimization problems. However, individual agents may raise concerns about sharing their internal data, simulation models, and decision models in collaborative optimization. Distributed simulation (DS) and federated learning have been widely used as privacy-preserving methods to hide simulation details and maintain data and model privacy. Despite their benefits, these methods often require large amounts of interaction and data to converge, which leads to a high communication time, especially if the agents are distributed around the world. To address this issue, we propose a distributed surrogate model for DS-based federated MaRL to utilize the surrogate model instead of DS during the training. This can enhance data efficiency and effectiveness to accelerate agent learning while maintaining data and model privacy. An aerospace supply chain (SC) is used as the experimental scenario to evaluate the performance of our proposed approach, in terms of SC profits, training convergence, and execution time. Experimental results show that our proposed approach can achieve higher SC profits with the same number of simulation runs, converge faster, and reduce execution time to gain the same level of SC profits.