In order to solve the problem that, in complex and wide traffic scenes, the accuracy and speed of multi-object detection can hardly be balanced by the existing object detection algorithms that are based on deep learning and big data, we improve the object detection framework SSD (Single Shot Multi-box Detector) and propose a new detection framework AP-SSD (Adaptive Perceive). We design a feature extraction convolution kernel library composed of multi-shape Gabor and color Gabor and then we train and screen the optimal feature extraction convolution kernel to replace the low-level convolution kernel of the original network to improve the detection accuracy. After that, we combine the single image detection framework with convolution long-term and short-term memory networks and by using the Bottle Neck-LSTM memory layer to refine and propagate the feature mapping between frames, we realize the temporal association of network frame-level information, reduce the calculation cost, succeed in tracking and identifying the targets affected by strong interference in video and reduce the missed alarm rate and false alarm rate by adding an adaptive threshold strategy. Moreover, we design a dynamic region amplification network framework to improve the detection and recognition accuracy of low-resolution small objects. Therefore, experiments on the improved AP-SSD show that this new algorithm can achieve better detection results when small objects, multiple objects, cluttered background and large-area occlusion are involved, thus ensuring this algorithm a good engineering application prospect.

Existing video captioning methods usually ignore the important fine-grained semantic attributes, the video diversity, as well as the association and motion state between objects within and between frames. Thus, they cannot adapt to small sample data sets. To solve the above problems, this paper proposes a novel video captioning model and an adversarial reinforcement learning strategy. Firstly, an object-scene relational graph model is designed based on the object detector and scene segmenter to express the association features. The graph is encoded by the graph neural network to enrich the expression of visual features. Meanwhile, a trajectory-based feature representation model is designed to replace the previous data-driven method to extract motion and attribute information, so as to analyze the object motion in the time domain and establish the connection between the visual content and language under small data sets. Finally, an adversarial reinforcement learning strategy and a multi- branch discriminator are designed to learn the relationship between the visual content and corresponding words so that rich language knowledge is integrated into the model. Experimental results on three standard datasets and one small sample dataset indicate that our proposed method achieves state-of-the-art performance. Also, the ablation experiments and visualization results verify the effectiveness of proposed each strategy.

Deep-learning convolutional neural networks (CNNs) have proven to be successful in various cognitive applications with a multilayer structure. The high computational energy and time requirements hinder the practical application of CNNs; hence, the realization of a highly energy-efficient and fast-learning neural network has aroused interest. In this work, we address the computing-resource-saving problem by developing a deep model, termed the Gabor convolutional neural network (Gabor CNN), which incorporates highly expression-efficient Gabor kernels into CNNs. In order to effectively imitate the structural characteristics of traditional weight kernels, we improve upon the traditional Gabor filters, having stronger frequency and orientation representations. In addition, we propose a procedure to train Gabor CNNs, termed the fast training method (FTM). In FTM, we design a new training method based on the multipopulation genetic algorithm (MPGA) and evaluation structure to optimize improved Gabor kernels, but train the rest of the Gabor CNN parameters with back-propagation. The training of improved Gabor kernels with MPGA is much more energy-efficient with less samples and iterations. Simple tasks, like character recognition on the Mixed National Institute of Standards and Technology database (MNIST), traffic sign recognition on the German Traffic Sign Recognition Benchmark (GTSRB), and face detection on the Olivetti Research Laboratory database (ORL), are implemented using LeNet architecture. The experimental result of the Gabor CNN and MPGA training method shows a 17–19% reduction in computational energy and time and an 18–21% reduction in storage requirements with a less than 1% accuracy decrease. We eliminated a significant fraction of the computation-hungry components in the training process by incorporating highly expression-efficient Gabor kernels into CNNs.

With the aim of automatic recognition of weak faults in hydraulic systems, this paper proposes an identification method based on multi-scale permutation entropy feature extraction of fault-sensitive intrinsic mode function (IMF) and deep belief network (DBN). In this method, the leakage fault signal is first decomposed by empirical mode decomposition (EMD), and fault-sensitive IMF components are screened by adopting the correlation analysis method. The multi-scale entropy feature of each screened IMF is then extracted and features closely related to the weak fault information are then obtained. Finally, DBN is used for identification of fault diagnosis. Experimental results prove that this identification method has an ideal recognition effect. It can accurately judge whether there is a leakage fault, determine the degree of severity of the fault, and can diagnose and analyze hydraulic weak faults in general.

Benefits from the powerful local modeling capability of deep convolutional networks (CNNs), remote sensing image change detection (CD) has made significant progress. In recent years, the rise of Transformers has further driven improvements in global feature extraction for bitemporal remote sensing images. Some prior efforts have tried to integrate CNN and Transformer, but they suffer from the limitation of inefficiently aggregating local features and contextual information. Besides, they struggle to refine change boundaries and exhibit inferior performance in detecting multi-scale and subtle changes. To tackle these interrelated problems, we propose a difference enhanced pyramid pooling dynamic sparse attention network (DPDANet) for CD, which integrates the potential of CNN and pyramid pooling dynamic sparse attention (PDSA) mechanism. Specifically, a pre-trained EfficientNetV2-S network is first used to extract multilevel local fine-grained features. Then, a global semantic enhancement network based on well-designed PDSA mechanism is proposed to extract rich global contextual information. The proposed difference enhancement module (DEM) combines long short term memory (LSTM) and deformable convolution to emphasize relevant and irrelevant changes, capturing precise boundary details of the changing region. A decoder is then employed for step-by-step upsampling of encoded features, with skip connections between local multi-scale features and globally enhanced features. Expensive experiments on four public CD datasets demonstrate that DPDANet outperforms state-of-the-art (SOTA) methods by reducing missed detections and false detections and achieving more accurate boundaries of the changing area.

Aiming at the real-time detection of multiple objects and micro-objects in large-scene remote sensing images, a cascaded convolutional neural network real-time object-detection framework for remote sensing images is proposed, which integrates visual perception and convolutional memory network reasoning. The detection framework is composed of two fully convolutional networks, namely, the strengthened object self-attention pre-screening fully convolutional network (SOSA-FCN) and the object accurate detection fully convolutional network (AD-FCN). SOSA-FCN introduces a self-attention module to extract attention feature maps and constructs a depth feature pyramid to optimize the attention feature maps by combining convolutional long-term and short-term memory networks. It guides the acquisition of potential sub-regions of the object in the scene, reduces the computational complexity, and enhances the network’s ability to extract multi-scale object features. It adapts to the complex background and small object characteristics of a large-scene remote sensing image. In AD-FCN, the object mask and object orientation estimation layer are designed to achieve fine positioning of candidate frames. The performance of the proposed algorithm is compared with that of other advanced methods on NWPU_VHR-10, DOTA, UCAS-AOD, and other open datasets. The experimental results show that the proposed algorithm significantly improves the efficiency of object detection while ensuring detection accuracy and has high adaptability. It has extensive engineering application prospects.