With the recent success of the deep neural networks (DNNs) in the field of artificial intelligence, the urge of deploying DNNs has drawn tremendous attention because it can benefit a wide range of applications on edge or embedded devices. Lightweight deep learning (DL) indicates the procedures of compressing DNN models into more compact ones, which are suitable to be executed on edge devices due to their limited resources and computational capabilities while maintaining comparable performance as the original. Currently, the approaches of model compression include but not limited to network pruning, quantization, knowledge distillation, neural architecture search. In this work, we present a fresh overview to summarize recent development and challenges for model compression.
1 July 2017 in International Journal of Computer Vision
Learning a discriminative model to distinguish a target from its surrounding distractors is essential to generic visual object tracking. Dynamic target representation adaptation against distractors is challenging due to the limited discriminative capabilities of prevailing trackers. We present a new visual Prompting mechanism for generic Visual Object Tracking (PiVOT) to address this issue. PiVOT proposes a prompt generation network with the pre-trained foundation model CLIP to automatically generate and refine visual prompts, enabling the transfer of foundation model knowledge for tracking. While CLIP offers broad category-level knowledge, the tracker, trained on instance-specific data, excels at recognizing unique object instances. Thus, PiVOT first compiles a visual prompt highlighting potential target locations. To transfer the knowledge of CLIP to the tracker, PiVOT leverages CLIP to refine the visual prompt based on the similarities between candidate objects and the reference templates across potential targets. Once the visual prompt is refined, it can better highlight potential target locations, thereby reducing irrelevant prompt information. With the proposed prompting mechanism, the tracker can generate improved instance-aware feature maps through the guidance of the visual prompt, thus effectively reducing distractors. The proposed method does not involve CLIP during training, thereby keeping the same training complexity and preserving the generalization capability of the pretrained foundation model. Extensive experiments across multiple benchmarks indicate that PiVOT, using the proposed prompting method can suppress distracting objects and enhance the tracker. The code for this work is publicly available.
Popularity prediction using social media is an important task because of its wide range of real-world applications such as advertisements, recommendation systems, and trend analysis. However, this task is challenging because social media is affected by multiple factors that cannot be easily modeled (e.g. quality of content, relevance to viewers, real-life events). Usually, other methods adopt the greedy approach to include as many modalities and factors as possible into their model but treat these features equally. To solve this phenomenon, our proposed method leverages the self-attention mechanism to effectively and automatically fuse different features to achieve better performance for the popularity prediction of a post, where the features used in our model can be mainly categorized into two modalities, semantic (text) and numeric features. With extensive experiments and ablation studies on the training and testing data of the challenging ACM Multimedia SMPD 2020 Challenge dataset, the evaluation results demonstrate the effectiveness of the proposed approach as compared with other methods.
Online healthcare applications have grown more popular over the years. For instance, telehealth is an online healthcare application that allows patients and doctors to schedule consultations, prescribe medication, share medical documents, and monitor health conditions conveniently. Apart from this, telehealth can also be used to store a patient's personal and medical information. With its rise in usage due to COVID-19, given the amount of sensitive data it stores, security measures are necessary. A simple way of making these applications more secure is through user authentication. One of the most common and often used authentications is face recognition. It is convenient and easy to use. However, face recognition systems are not foolproof. They are prone to malicious attacks like printed photos, paper cutouts, replayed videos, and 3D masks. The goal of face anti-spoofing is to differentiate real users (live) from attackers (spoof). Although effective in terms of performance, existing methods use a significant amount of parameters, making them resource-heavy and unsuitable for handheld devices. Apart from this, they fail to generalize well to new environments like changes in lighting or background. This paper proposes a lightweight face anti-spoofing framework that does not compromise on performance. Our proposed method achieves good performance with the help of an ArcFace Classifier (AC). The AC encourages differentiation between spoof and live samples by making clear boundaries between them. With clear boundaries, classification becomes more accurate. We further demonstrate our model's capabilities by comparing the number of parameters, FLOPS, and performance with other state-of-the-art methods.