This page has only limited features, please log in for full access.
The deep convolutional neural network has made significant progress in cloud detection. However, the compromise between having a compact model and high accuracy has always been a challenging task in cloud detection for large-scale remote sensing imagery. A promising method to tackle this problem is knowledge distillation, which usually lets the compact model mimic the cumbersome model’s output to get better generalization. However, vanilla knowledge distillation methods cannot properly distill the characteristics of clouds in remote sensing images. In this paper, we propose a novel self-attention knowledge distillation approach for compact and accurate cloud detection, named Bidirectional Self-Attention Distillation (Bi-SAD). Bi-SAD lets a model learn from itself without adding additional parameters or supervision. With bidirectional layer-wise features learning, the model can get a better representation of the cloud’s textural information and semantic information, so that the cloud’s boundaries become more detailed and the predictions become more reliable. Experiments on a dataset acquired by GaoFen-1 satellite show that our Bi-SAD has a great balance between compactness and accuracy, and outperforms vanilla distillation methods. Compared with state-of-the-art cloud detection models, the parameter size and FLOPs are reduced by 100 times and 400 times, respectively, with a small drop in accuracy.
Yajie Chai; Kun Fu; Xian Sun; Wenhui Diao; Zhiyuan Yan; Yingchao Feng; Lei Wang. Compact Cloud Detection with Bidirectional Self-Attention Knowledge Distillation. Remote Sensing 2020, 12, 2770 .
AMA StyleYajie Chai, Kun Fu, Xian Sun, Wenhui Diao, Zhiyuan Yan, Yingchao Feng, Lei Wang. Compact Cloud Detection with Bidirectional Self-Attention Knowledge Distillation. Remote Sensing. 2020; 12 (17):2770.
Chicago/Turabian StyleYajie Chai; Kun Fu; Xian Sun; Wenhui Diao; Zhiyuan Yan; Yingchao Feng; Lei Wang. 2020. "Compact Cloud Detection with Bidirectional Self-Attention Knowledge Distillation." Remote Sensing 12, no. 17: 2770.