12 January 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Modern CNN-based object detectors assign anchors for ground-truth objects under the restriction of object-anchor Intersection-over-Union (IoU). In this study, we propose a learning-to-match (LTM) method to break IoU restriction, allowing objects to match anchors in a flexible manner. LTM updates hand-crafted anchor assignment to “free” anchor matching by formulating detector training in the Maximum Likelihood Estimation (MLE) framework. During the training phase, LTM is implemented by converting the detection likelihood to anchor matching loss functions which are plug-and-play. Minimizing the matching loss functions drives learning and selecting features which best explain a class of objects with respect to both classification and localization. LTM is extended from anchor-based detectors to anchor-free detectors, validating the general applicability of learnable object-feature matching mechanism for visual object detection. Experiments on MS COCO dataset demonstrate that LTM detectors consistently outperform counterpart detectors with significant margins. The last but not the least, LTM requires negligible computational cost in both training and inference phases as it does not involve any additional architecture or parameter. Code has been made publicly available.
16 May 2019 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Convolutional neural networks (CNNs) are widely used and have achieved great success in computer vision and speech processing applications. However, deploying the large-scale CNN model in the embedded system is subject to the constraints of computation and memory. An optimized block-floating-point (BFP) arithmetic is adopted in our accelerator for efficient inference of deep neural networks in this paper. The feature maps and model parameters are represented in 16-bit and 8-bit formats, respectively, in the off-chip memory, which can reduce memory and off-chip bandwidth requirements by 50% and 75% compared to the 32-bit FP counterpart. The proposed 8-bit BFP arithmetic with optimized rounding and shifting-operation-based quantization schemes improves the energy and hardware efficiency by three times. One CNN model can be deployed in our accelerator without retraining at the cost of an accuracy loss of not more than 0.12%. The proposed reconfigurable accelerator with three parallelism dimensions, ping-pong off-chip DDR3 memory access, and an optimized on-chip buffer group is implemented on the Xilinx VC709 evaluation board. Our accelerator achieves a performance of 760.83 GOP/s and 82.88 GOP/s/W under a 200-MHz working frequency, significantly outperforming previous accelerators.
23 November 2022 in IEEE Communications Surveys & Tutorials
Sixth-generation (6G) cellular systems will have an inherent vulnerability to physical (PHY)-layer attacks and privacy leakage, due to the large-scale heterogeneous networks with booming time-sensitive applications. Important wireless techniques including non-orthogonal multiple access, mobile edge computing, millimeter-wave, massive multiple-input and multiple-output, visible light communication, terahertz, and intelligent reflecting surface can improve the spectrum efficiency and quality-of-service but will raise challenges for the 6G PHY and cross-layer security and privacy protection. Existing optimization based PHY and cross-layer security and privacy protection schemes such as the convex optimization method have to rely on accurate attack patterns and strategies and thus suffer from performance degradation in 6G systems that have shorter communication latency, more devices and higher spectrum efficiency than 5G. Reinforcement learning (RL) algorithms help wireless devices optimize their security policies to enhance the security performance in dynamic networks against smart attacks without depending on the attack model. Therefore, this article provides a comprehensive survey on the RL based 6G PHY cross-layer security and privacy protection. In this article, we investigate the potential attacks in 6G systems and discuss the PHY cross-layer security solutions. A brief overview of reinforcement learning algorithms is provided. Afterward, we review the RL based PHY-layer security and privacy protection and discuss how to apply RL algorithms in 6G security scenarios, especially focusing on the game with jammers, eavesdroppers, spoofers and inference attackers. The RL based security solutions for unmanned aerial vehicles (UAVs) and cross-layer scenarios are also reviewed. The future research directions are identified and the corresponding RL based potential solutions are discussed for 6G.
7 February 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Supervised deep learning has achieved tremendous success in many computer vision tasks, which however is prone to overfit noisy labels. To mitigate the undesirable influence of noisy labels, robust loss functions offer a feasible approach to achieve noise-tolerant learning. In this work, we systematically study the problem of noise-tolerant learning with respect to both classification and regression. Specifically, we propose a new class of loss function, namely asymmetric loss functions (ALFs), which are tailored to satisfy the Bayes-optimal condition and thus are robust to noisy labels. For classification, we investigate general theoretical properties of ALFs on categorical noisy labels, and introduce the asymmetry ratio to measure the asymmetry of a loss function. We extend several commonly-used loss functions, and establish the necessary and sufficient conditions to make them asymmetric and thus noise-tolerant. For regression, we extend the concept of noise-tolerant learning for image restoration with continuous noisy labels. We theoretically prove that $\ell _{p}$ loss ( $p>0$ ) is noise-tolerant for targets with the additive white Gaussian noise. For targets with general noise, we introduce two losses as surrogates of $\ell _{0}$ loss that seeks the mode when clean pixels keep dominant. Experimental results demonstrate that ALFs can achieve better or comparative performance compared with the state-of-the-arts.
17 December 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
6D object pose estimation is a fundamental yet challenging problem in computer vision. Convolutional Neural Networks (CNNs) have recently proven to be capable of predicting reliable 6D pose estimates even under monocular settings. Nonetheless, CNNs are identified as being extremely data-driven, and acquiring adequate annotations is oftentimes very time-consuming and labor intensive. To overcome this limitation, we propose a novel monocular 6D pose estimation approach by means of self-supervised learning, removing the need for real annotations. After training our proposed network fully supervised with synthetic RGB data, we leverage current trends in noisy student training and differentiable rendering to further self-supervise the model on these unsupervised real RGB(-D) samples, seeking for a visually and geometrically optimal alignment. Moreover, employing both visible and amodal mask information, our self-supervision becomes very robust towards challenging scenarios such as occlusion. Extensive evaluations demonstrate that our proposed self-supervision outperforms all other methods relying on synthetic data or employing elaborate techniques from the domain adaptation realm. Noteworthy, our self-supervised approach consistently improves over its synthetically trained baseline and often almost closes the gap towards its fully supervised counterpart.