In this work we explore enhancing performance of reinforcement learning algorithms in video game environments by feeding it better, more relevant data. For this purpose, we use semantic segmentation to transform the images that would be used as input for the reinforcement learning algorithm from their original domain to a simplified semantic domain with just silhouettes and class labels instead of textures and colors, and then we train the reinforcement learning algorithm with these simplified images. We have conducted different experiments to study multiple aspects: feasibility of our proposal, and potential benefits to model generalization and transfer learning. Experiments have been performed with the Super Mario Bros video game as the testing environment. Our results show multiple advantages for this method. First, it proves that using semantic segmentation enables reaching higher performance than the baseline reinforcement learning algorithm without modifying the actual algorithm, and in fewer episodes; second, it shows noticeable performance improvements when training on multiple levels at the same time; and finally, it allows to apply transfer learning for models trained on visually different environments. We conclude that using semantic segmentation can certainly help reinforcement learning algorithms that work with visual data, by refining it. Our results also suggest that other computer vision techniques may also be beneficial for data prepossessing. Models and code will be available on github upon acceptance.

Semantic segmentation is a computer vision task where classification is performed at the pixel level. Due to this, the process of labeling images for semantic segmentation is time-consuming and expensive. To mitigate this cost there has been a surge in the use of synthetically generated data—usually created using simulators or videogames—which, in combination with domain adaptation methods, can effectively learn how to segment real data. Still, these datasets have a particular limitation: due to their closed-set nature, it is not possible to include novel classes without modifying the tool used to generate them, which is often not public. Concurrently, generative models have made remarkable progress, particularly with the introduction of diffusion models, enabling the creation of high-quality images from text prompts without additional supervision. In this work, we propose an unsupervised pipeline that leverages Stable Diffusion and Segment Anything Module to generate class examples with an associated segmentation mask, and a method to integrate generated cutouts for novel classes in semantic segmentation datasets, all with minimal user input. Our approach aims to improve the performance of unsupervised domain adaptation methods by introducing novel samples into the training data without modifications to the underlying algorithms. With our methods, we show how models can not only effectively learn how to segment novel classes, with an average performance of 51% intersection over union for novel classes, but also reduce errors for other, already existing classes, reaching a higher performance level overall.

Objective. Image-processing-based remote photoplethysmography algorithms are usually composed of steps where different methods are used, and often, researchers perform these steps using methods that are not necessarily the best for their application. With our toolkit, we want to provide easy and fast access to different state-of-the-art methods for the most common image-processing steps in remote photoplethysmography algorithms. Methods. Our toolkit was programmed in Python and was developed with sequential workflow in mind, making it versatile and easy to use in interactive environments. It also includes tools so the users can modify or extend it if they want to, and will be updated as new methods for the different steps are published. Results. Our use case examples and validation show an effective approach and how the toolkit can be used for exhaustive evaluation and ablation studies in a simple way. We also show how choosing different methods can affect the final heart rate estimation accuracy at the cost of computation time. Conclusion. With this toolkit we are providing researchers with a versatile, easy-to-use tool, with access to different methods for the most common steps in remote photoplethysmography algorithms. Significance. Our toolkit is a relevant tool for researchers in the remote photoplethysmography field due to their versatility, ease of use, and adaptability. (It will be available on https://github.com/Montyro/rppgtkgithub upon acceptance).