The manner of walking (gait) is a powerful biometric that is used as a unique fingerprinting method, allowing unobtrusive behavioral analytics to be performed at a distance without subject cooperation. As opposed to more traditional biometric authentication methods, gait analysis does not require explicit cooperation of the subject and can be performed in low-resolution settings, without requiring the subject’s face to be unobstructed/clearly visible. Most current approaches are developed in a controlled setting, with clean, gold-standard annotated data, which powered the development of neural architectures for recognition and classification. Only recently has gait analysis ventured into using more diverse, large-scale, and realistic datasets to pretrained networks in a self-supervised manner. Self-supervised training regime enables learning diverse and robust gait representations without expensive manual human annotations. Prompted by the ubiquitous use of the transformer model in all areas of deep learning, including computer vision, in this work, we explore the use of five different vision transformer architectures directly applied to self-supervised gait recognition. We adapt and pretrain the simple ViT, CaiT, CrossFormer, Token2Token, and TwinsSVT on two different large-scale gait datasets: GREW and DenseGait. We provide extensive results for zero-shot and fine-tuning on two benchmark gait recognition datasets, CASIA-B and FVG, and explore the relationship between the amount of spatial and temporal gait information used by the visual transformer. Our results show that in designing transformer models for processing motion, using a hierarchical approach (i.e., CrossFormer models) on finer-grained movement fairs comparatively better than previous whole-skeleton approaches.

The orientation of objects plays an important role in accurate predictions for the tasks of classification, detection, and trajectory estimation. This is especially important in the automotive domain, where estimating an accurate car orientation can significantly impact the effectiveness of the other prediction tasks. This work presents Car Full View (CFV), a novel dataset for car orientation prediction from images obtained by video recording all possible angles of individual vehicles in diverse scenarios. We developed a tool to semi-automatically annotate all the video frames with the respective car angle based on the walking speed of the recorder and manually annotated key angles. The final dataset contains over 23,000 images of individual cars along with fine-grained angle annotations. We study the performance of three state-of-the-art deep learning architectures on this dataset in three different learning settings: classification, regression, and multi-objective. The top result of 3.39° in circular mean absolute error (CMAE) shows that the model accurately predicts car orientations for unseen vehicles and images. Furthermore, we test the trained models on images from two different datasets and show their generalization capability to realistic images. We release the dataset and the best models while publishing a web service to annotate new images.

Accurately predicting vehicle attributes from an image is a challenging yet crucial task with numerous applications in the automotive industry, including car identification, price estimation, and traffic monitoring. Traditional Computer Vision approaches have primarily focused on single-objective tasks where the model either classifies a single feature of the vehicle or predicts a class that aggregates multiple characteristics. These methods often overlook the inter-dependencies between different vehicle attributes and may not fully capture the fine-grained separation of information that Deep Learning models should learn. To address these limitations, we propose partitioning vehicle characteristics into a hierarchical structure where information at an upper level (supertask) conditions the information at the lower level (subtask). To leverage this hierarchical structure, we propose a multi-task neural network architecture that conditions the predictions of subtasks based on its prediction for the supertask using attention and gating mechanisms. This conditioning is further reinforced by a proposed task-consistency loss which is designed to teach the model the task hierarchy in scenarios where the training data is limited. We evaluate our approach on multiple image-based vehicle classification benchmarks, including our proposed CarModelDataset, a difficult dataset specifically collected for a comprehensive analysis. Our proposed methodology and architecture obtain an F1 score of 97.14% on Stanford Cars, 85.08% on Autovit, and 63.03% on CarModelDataset. Our results show that the hierarchical conditioning approach outperforms single-task models in all scenarios, especially in contexts with limited data, highlighting the effectiveness of our method. We release the code^a and our best performing model for research purposes, and make the CarModelDataset available upon request.