This review explores the integration of enhanced personalization and seamless multimodal interfaces in the field of fashion design and recommendation. We examine the increasing demand for personalized fashion experiences and the potential of multimodal interfaces in facilitating effective communication between designers and users. By leveraging user preferences, body measurements, and style choices, artificial intelligence (AI) systems can deliver highly personalized fashion recommendations. The integration of various input modalities, including text, images and sketches, enables designers and users to communicate their design ideas with ease. The primary results highlight the transformative potential of enhanced personalization and seamless multimodal interfaces, empowering designers and consumers to co-create unique and personalized designs. This paradigm shift fosters a deeper level of engagement and creativity within the fashion industry. Embracing this advancement unlocks unprecedented opportunities for designers, brands, and consumers, ushering in a new era of innovation and creativity in fashion design.

Image-based fashion design with AI techniques has attracted increasing attention in recent years. We focus on the reference-based fashion design task, where we aim to combine a reference appearance image and a clothing image to generate a new fashion clothing image. Although existing diffusion-based image translation methods have enabled flexible style transfer, it is often difficult to transfer the appearance of the image realistically during reverse diffusion. When the referenced appearance domain greatly differs from the source domain, it often leads to the collapse in the translation. To tackle this issue, we present a novel diffusion model-based unsupervised structureaware transfer method, namely DiffFashion. Our method is free of model tuning and structure-preserving and has high flexibility in transferring from images with large domain gaps. Specifically, based on the optimal transport properties, we keep a shared latent across the clothing image and reference appearance image to bridge the gap between the two domains in the denoising process, and the latent of the reference image is gradually adapted to the clothing domain. Simultaneously, the structure is transferred from the source clothing to the output fashion image with mixed guidance, including pre-trained Vision Transformer (ViT) guidance and a foreground mask guidance, to further preserve the structure and appearance semantics from source and reference images. Our experimental results show that the proposed method outperforms state-of-the-art baseline models, generating more realistic images in the fashion design task. Code and demo can be found at https://github.com/Rem105- 210/DiffFashion .

Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific vision tasks. Yet, existing methods either employ complex spatial-temporal modules or rely heavily on additional perception models to extract temporal features for video understanding, performing well only on short videos. For long videos, the computational complexity and memory costs associated with long-term temporal connections are significantly increased, posing additional challenges. Leveraging the hierarchical memory structure of the Atkinson-Shiffrin memory model, with tokens in Transformers being employed as the carriers of memory in combination, we propose MovieChat within a training-free memory consolidation mechanism to overcome these challenges, which transfers dense frames from short-term memory into sparse tokens in long-term memory by temporally merging adjacent frames. We lift pre-trained large multi-modal models for understanding long videos without additional trainable modules, employing a zero-shot approach. Additionally, in our new version, MovieChat+, we design an enhanced training-free vision-question matching-based memory consolidation mechanism to better anchor predictions to relevant visual content. MovieChat achieves state-of-the-art performance in long video understanding, along with the released MovieChat-1 K benchmark with 1 K long video, 2 K temporal grounding labels, and 14 K manual annotations.

Multi-object tracking (MOT) is an essential task in the computer vision field. With the fast development of deep learning technology in recent years, MOT has achieved great improvement. However, some challenges still remain, such as sensitiveness to occlusion, instability under different lighting conditions, and non-robustness to deformable objects, causing incorrect temporal associations. To address such common challenges in most of the existing trackers, in this paper, a tracklet booster (TBooster) algorithm is proposed to correct the association errors resulting from existing trackers. The correction of the association error from TBooster has two folds: split tracklets on potential ID-change positions and then connect multiple tracklets into one if they are from the same object. To achieve this goal, the TBooster consists of two components, i.e., Splitter and Connector. In Splitter, an architecture with stacked temporal dilated convolution blocks is employed for the splitting position prediction via label smoothing strategy with adaptive Gaussian kernels. In Connector, a multi-head self-attention-based encoder is exploited for the tracklet embedding, which is further used to connect tracklets into full tracks. We conduct sufficient experiments on MOT17 and MOT20 benchmark datasets and achieve promising results. Combined with the proposed tracklet booster, existing trackers can achieve large improvements on the IDF1 score, which shows the effectiveness of the proposed TBooster.

Accurately delineating individual teeth and the gingiva in the three-dimension (3D) intraoral scanned (IOS) mesh data plays a pivotal role in many digital dental applications, e.g., orthodontics. Recent research shows that deep learning based methods can achieve promising results for 3D tooth segmentation, however, most of them rely on high-quality labeled dataset which is usually of small scales as annotating IOS meshes requires intensive human efforts. In this paper, we propose a novel self-supervised learning framework, named STSNet, to boost the performance of 3D tooth segmentation leveraging on large-scale unlabeled IOS data. The framework follows two-stage training, i.e., pre-training and fine-tuning. In pre-training, three hierarchical-level, i.e., point-level, region-level, cross-level, contrastive losses are proposed for unsupervised representation learning on a set of predefined matched points from different augmented views. The pre-trained segmentation backbone is further fine-tuned in a supervised manner with a small number of labeled IOS meshes. With the same amount of annotated samples, our method can achieve an mIoU of 89.88%, significantly outperforming the supervised counterparts. The performance gain becomes more remarkable when only a small amount of labeled samples are available. Furthermore, STSNet can achieve better performance with only 40% of the annotated samples as compared to the fully supervised baselines. To the best of our knowledge, we present the first attempt of unsupervised pre-training for 3D tooth segmentation, demonstrating its strong potential in reducing human efforts for annotation and verification.

Real-time road quality monitoring, involves using technologies to collect data on the conditions of the road, including information on potholes, cracks, and other defects. This information can help to improve safety for drivers and reduce costs associated with road damage. Traditional methods are time-consuming and expensive, leading to limited spatial coverage and delayed responses to road conditions. With the widespread use of smartphones and ubiquitous computing technologies, data can be collected from built-in sensors of mobile phones and in-vehicle video, on a large scale. This has raised the question of how these data can be used for road pothole detection and has significant practical relevance. Current methods either use acceleration sequence classification techniques, or image recognition techniques based on deep learning. However, accelerometer-based detection has limited coverage and is sensitive to the driving speed, while image recognition-based detection is highly affected by ambient light. To address these issues, this study proposes a method that utilizes the fusion of accelerometer data and in-vehicle video data, which is uploaded by the participating users. The preprocessed accelerometer data and intercepted video frames, were then encoded into real-valued vectors, and projected into the public space. A deep learning-based training approach was used to learn from the public space and identify road anomalies. Spatial density-based clustering was implemented in a multi-vehicle scenario, to improve reliability and optimize detection results. The performance of the model is evaluated with confusion matrix-based classification metrics. Real-world vehicle experiments are carried out, and the results demonstrate that the proposed method can improve accuracy by 6% compared to the traditional method. Consequently, the proposed method provides a novel approach for large-scale pavement anomaly detection.