Due to its great application value in the military and civilian fields, ship detection in synthetic aperture radar (SAR) images has always attracted much attention. However, ship targets in High-Resolution (HR) SAR images show the significant characteristics of multi-scale, arbitrary directions and dense arrangement, posing enormous challenges to detect ships quickly and accurately. To address these issues above, a novel YOLO-based arbitrary-oriented SAR ship detector using bi-directional feature fusion and angular classification (BiFA-YOLO) is proposed in this article. First of all, a novel bi-directional feature fusion module (Bi-DFFM) tailored to SAR ship detection is applied to the YOLO framework. This module can efficiently aggregate multi-scale features through bi-directional (top-down and bottom-up) information interaction, which is helpful for detecting multi-scale ships. Secondly, to effectively detect arbitrary-oriented and densely arranged ships in HR SAR images, we add an angular classification structure to the head network. This structure is conducive to accurately obtaining ships’ angle information without the problem of boundary discontinuity and complicated parameter regression. Meanwhile, in BiFA-YOLO, a random rotation mosaic data augmentation method is employed to suppress the impact of angle imbalance. Compared with other conventional data augmentation methods, the proposed method can better improve detection performance of arbitrary-oriented ships. Finally, we conduct extensive experiments on the SAR ship detection dataset (SSDD) and large-scene HR SAR images from GF-3 satellite to verify our method. The proposed method can reach the detection performance with precision = 94.85%, recall = 93.97%, average precision = 93.90%, and F1-score = 0.9441 on SSDD. The detection speed of our method is approximately 13.3 ms per 512 × 512 image. In addition, comparison experiments with other deep learning-based methods and verification experiments on large-scene HR SAR images demonstrate that our method shows strong robustness and adaptability.

With the advent of the era of high-resolution remote sensing, semantic segmentation methods for solving pixel-level classification have been widely studied. Deep learning has significantly advanced deep feature extraction methods, becoming widely employed in remote sensing image analysis. Deep feature fusion methods is able to effectively combine features from different sources. Optical and synthetic aperture radar (SAR) images stand out as primary data sources in remote sensing, offering complementary and consistent information. Fusion of deep semantic features of optical and SAR images can alleviate the limitations of single-source images in application and improve semantic segmentation accuracy. Therefore, this article reviews the research on deep fusion of optical and SAR images in semantic segmentation tasks from four aspects. Firstly, we provide a summary of challenges and research methods pertinent to semantic segmentation of remote sensing images. Then the challenges and urgent needs of deep feature fusion of optical and SAR images are analyzed, and current research is summarized from the perspective of structural design by studying various feature fusion strategies. Additionally, the compilation and in-depth analysis of open-source optical and SAR datasets suitable for semantic segmentation are undertaken, serving as fundamental resources for future research endeavors. Finally, the article identifies the major challenges summarized from the literature review in this field, outlining expectations and potential future directions for researchers.

In practical SAR target detection applications, it is often encountered that the training and testing data come from different SAR sensors, leading to a decline in SAR target detection performance. Although domain adaptation methods can achieve model generalization through feature transfer, the change of scattering characteristics for the same target and the difference of feature distribution, caused by cross-sensor, cannot be ignored in SAR images. It is inevitable to lead to the escalation of the offset in the bounding box regression and deviation of the feature alignment. To address these issues, a cross-sensor SAR image target detection method based on dynamic feature discrimination and center-aware calibration is proposed. Based on the domain adaptation framework, initially, a Dynamic Feature Discrimination Module (DFDM) is introduced to address the exacerbated offset in the regression. A bidirectional spatial feature aggregation mechanism is employed to aggregate features in both horizontal and vertical directions and a multi-scale structure is adopted to enhance the scattering and semantic features, which can dynamically constrain the target position while improving target discrimination capability. Then, the Center-Aware Calibration Module (CACM) is designed to address the alignment deviation in feature transfer. The target salience relationship is modeled based on the distance between different positions and the target center to suppress background clutter interference. The perception center of the target is focused by combining the centerness map and classification map, which can calibrate the domain-invariant features and alleviate misalignment. Finally, the proposed method is tested on two datasets, MiniSAR and FARAD, and compared with the latest domain adaption methods. Both mAP and F1 values have improved by more than 6%-20%, verifying the effectiveness of the proposed method.

Visual speech, referring to the visual domain of speech, has attracted increasing attention due to its wide applications, such as public security, medical treatment, military defense, and film entertainment. As a powerful AI strategy, deep learning techniques have extensively promoted the development of visual speech learning. Over the past five years, numerous deep learning based methods have been proposed to address various problems in this area, especially automatic visual speech recognition and generation. To push forward future research on visual speech, this paper will present a comprehensive review of recent progress in deep learning methods on visual speech analysis. We cover different aspects of visual speech, including fundamental problems, challenges, benchmark datasets, a taxonomy of existing methods, and state-of-the-art performance. Besides, we also identify gaps in current research and discuss inspiring future research directions.

The extraction of a valuable set of features and the design of a discriminative classifier are crucial for target recognition in SAR image. Although various features and classifiers have been proposed over the years, target recognition under extended operating conditions (EOCs) is still a challenging problem, e.g., target with configuration variation, different capture orientations, and articulation. To address these problems, this paper presents a new strategy for target recognition. We first propose a low-dimensional representation model via incorporating multi-manifold regularization term into the low-rank matrix factorization framework. Two rules, pairwise similarity and local linearity, are employed for constructing multiple manifold regularization. By alternately optimizing the matrix factorization and manifold selection, the feature representation model can not only acquire the optimal low-rank approximation of original samples, but also capture the intrinsic manifold structure information. Then, to take full advantage of the local structure property of features and further improve the discriminative ability, local sparse representation is proposed for classification. Finally, extensive experiments on moving and stationary target acquisition and recognition (MSTAR) database demonstrate the effectiveness of the proposed strategy, including target recognition under EOCs, as well as the capability of small training size.