The field of neural style transfer (NST) has witnessed remarkable progress in the past few years, with approaches being able to synthesize artistic and photorealistic images and videos of exceptional quality. To evaluate such results, a diverse landscape of evaluation methods and metrics is used, including authors' opinions based on side‐by‐side comparisons, human evaluation studies that quantify the subjective judgements of participants, and a multitude of quantitative computational metrics which objectively assess the different aspects of an algorithm's performance. However, there is no consensus regarding the most suitable and effective evaluation procedure that can guarantee the reliability of the results. In this review, we provide an in‐depth analysis of existing evaluation techniques, identify the inconsistencies and limitations of current evaluation methods, and give recommendations for standardized evaluation practices. We believe that the development of a robust evaluation framework will not only enable more meaningful and fairer comparisons among NST methods but will also enhance the comprehension and interpretation of research findings in the field.
Temporal consistency and content preservation are the prominent challenges in artistic video style transfer. To address these challenges, we present a technique that utilizes depth data and we demonstrate this on real-world videos from the web, as well as on a standard video dataset of three-dimensional computer-generated content. Our algorithm employs an image-transformation network combined with a depth encoder network for stylizing video sequences. For improved global structure preservation and temporal stability, the depth encoder network encodes ground-truth depth information which is fused into the stylization network. To further enforce temporal coherence, we employ ConvLSTM layers in the encoder, and a loss function based on calculated depth information for the output frames is also used. We show that our approach is capable of producing stylized videos with improved temporal consistency compared to state-of-the-art methods whilst also successfully transferring the artistic style of a target painting.
The segmentation of fine-grained and complex structures from volumetric data, such as 3D biomedical images, is a manually intensive process, with performance hindered by limited training data and the difficulty of adapting AI models for specialised datasets. Here, we introduce SPROUT, a user-friendly and interpretable segmentation framework that leverages domain-specific priors, enabling experts to translate their knowledge into reproducible, high-quality segmentations across diverse imaging modalities without the need for training data. Its adaptive design facilitates parameter transferability and improves generalisation across similar datasets. Implemented as scripts and a napari plugin, it supports interactive editing and scalable batch processing, lowering the technical barrier for domain experts. We applied SPROUT to datasets spanning different imaging modalities, anatomical complexities, postures, and target structures, producing high-quality segmentations across 2D and 3D tasks. Quantitative comparisons with other methods on a representative dataset showed SPROUT achieved results comparable to expert-corrected interpolation while requiring substantially less manual input. In scenarios where SPROUT achieved high-quality results, supervised models often struggled to reach similar accuracy, highlighting the challenge of deep learning methods in complex domains. We also explored integration with foundation models to accelerate segmentation in high-contrast datasets, illustrating potential for hybrid workflows.
In the past decade, substantial global efforts have been devoted to the development of reliable and efficient solutions for early flood warning and monitoring. One of the most common strategies for tackling this challenge involves the application of computer vision techniques to images obtained from the numerous surveillance cameras present in urban settings today. While there are various datasets available for training and testing these techniques, none of them specifically addresses the issue of out-of-distribution (OoD) behavior. This issue becomes particularly critical when evaluating the reliability of these methods under challenging environmental conditions. Our work stands as the first attempt to bridge this gap by introducing a new, highly controlled synthetic dataset that encompasses the essential attributes required for analyzing OoD behavior. The very high correlation between the accuracy of artificial intelligence (AI) models trained on our synthetic dataset and models trained on real-world data proves our dataset’s ability to predict real-world OoD behavior reliably.
Artistic Neural Style Transfer (NST) has achieved remarkable success for images. However, this is not the case for dynamic 3D environments, such as computer games, where temporal coherence remains a challenge. Our paper presents an approach that uses the G-buffer information available in a game pipeline to generate robust and temporally consistent in-game artistic stylizations based on a style reference image. We use a synthetic dataset created from open-source computer games and demonstrate that the utilization of depth, normals, and edge information enables the stylization process to be more aware of the geometric and semantic aspects of a game scene. The proposed approach builds on previous work by injecting style transfer in the rendering pipeline, while also utilizing G-buffer information during inference time to improve upon the stability of the stylizations, offering a controllable way to stylize computer games in terms of temporal coherence and content preservation. Qualitative and quantitative evaluations of our in-game stylization network demonstrate significantly higher temporal stability compared to existing style transfer approaches when stylizing 3D computer games.