27 January 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Availability of labelled data is the major obstacle to the deployment of deep learning algorithms for computer vision tasks in new domains. The fact that many frameworks adopted to solve different tasks share the same architecture suggests that there should be a way of reusing the knowledge learned in a specific setting to solve novel tasks with limited or no additional supervision. In this work, we first show that such knowledge can be shared across tasks by learning a mapping between task-specific deep features in a given domain. Then, we show that this mapping function, implemented by a neural network, is able to generalize to novel unseen domains. Besides, we propose a set of strategies to constrain the learned feature spaces, to ease learning and increase the generalization capability of the mapping network, thereby considerably improving the final performance of our framework. Our proposal obtains compelling results in challenging synthetic-to-real adaptation scenarios by transferring knowledge between monocular depth estimation and semantic segmentation tasks.
17 July 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
In recent years, Neural Fields ( NF s) have emerged as an effective tool for encoding diverse continuous signals such as images, videos, audio, and 3D shapes. When applied to 3D data, NF s offer a solution to the fragmentation and limitations associated with prevalent discrete representations. However, given that NF s are essentially neural networks, it remains unclear whether and how they can be seamlessly integrated into deep learning pipelines for solving downstream tasks. This paper addresses this research problem and introduces nf2vec , a framework capable of generating a compact latent representation for an input NF in a single inference pass. We demonstrate that nf2vec effectively embeds 3D objects represented by the input NF s and showcase how the resulting embeddings can be employed in deep learning pipelines to successfully address various tasks, all while processing exclusively NF s. We test this framework on several NF s used to represent 3D surfaces, such as unsigned/signed distance and occupancy fields. Moreover, we demonstrate the effectiveness of our approach with more complex NF s that encompass both geometry and appearance of 3D objects such as neural radiance fields.
1 November 2022 in IEEE Transactions on Intelligent Transportation Systems
Vehicle viewpoint estimation from monocular images is a crucial component for autonomous driving vehicles and for fleet management applications. In this paper, we make several contributions to advance the state-of-the-art on this problem. We show the effectiveness of applying a smoothing filter to the output neurons of a Convolutional Neural Network (CNN) when estimating vehicle viewpoint. We point out the overlooked fact that, under the same viewpoint, the appearance of a vehicle is strongly influenced by its position in the image plane, which renders viewpoint estimation from appearance an ill-posed problem. We show how, by inserting in the model a CoordConv layer to provide the coordinates of the vehicle, we are able to solve such ambiguity and greatly increase performance. Finally, we introduce a new data augmentation technique that improves viewpoint estimation on vehicles that are closer to the camera or partially occluded. All these improvements let a lightweight CNN reach optimal results while keeping inference time low. An extensive evaluation on a viewpoint estimation benchmark () and on actual vehicle camera data () shows that our method significantly outperforms the state-of-the-art in vehicle viewpoint estimation, both in terms of accuracy and memory footprint.
10 April 2024 in IEEE Transactions on Intelligent Transportation Systems
The consequences of vehicle crashes are extremely costly, especially in industrial contexts, where the loss of income due to the vehicle unavailability while the incident is investigated adds to the damage produced by the event. The ongoing shift toward more connected vehicles, featuring sensors and cameras, offers the opportunity to alleviate such losses by speeding up the resolution of disputes. In this paper, we show how data routinely collected by connected vehicles can be fused to attain automatic reconstruction of the crash dynamic, a key element that has to be provided by drivers to submit a First Notification of Loss. We build upon state-of-the-art methods in areas such as SLAM, depth estimation and object detection to create a reconstruction of the scene with the vehicles involved localized both in space and time, which we present in an animated bird’s eye view. Our pipeline is evaluated on a challenging benchmark of real world videos and it is shown to create reliable reconstructions of the moment of the impact in more than 50% of scenes and overall good reconstructions in about 37% of them.