Multimodal fusion is an essential research area in computer vision application. Though, there are still many obstacles in image fusion domain, that cause the loss key content, due to method limitation or its least efficiency. To solve these problems, a novel method PFAF-Net (Pyramid Feature Attention Fusion strategy) based on multi-scale features with core idea of different level fusion strategy. Firstly, multiscale high-level features with different receptive fields are extracted by pyramid feature extraction module. Secondly, high level and low features are fused with different fusion techniques i.e. attention based fusion strategies are adopted to efficiently fused the global and local features separately. Finally, the fused image is reconstructed with the enhanced fused features by the decoder. Thus, the proposed method not only integrate multimodal data but also has rich content details inferring a superior image fusion. In addition, proposed PFAF-Net has better generalization ability than existing methods on four different multimodal (i.e. multi-modal medical, multi-exposure, multifocus and visual-infrared) of benchmark datasets. Compared with other state-of-the-art fusion methods experimentally, the proposed method has shown comparable or better fusion performance in both subjective and objective evaluation.

Change detection (CD) is an essential remote sensing application for Earth observations widely used for several monitoring, management, and surveillance-related purposes. Generally, pixel-to-pixel prediction roles are susceptible to position details, as high-resolution bitemporal images contain abundant ground details, and so it needs more precautions to extract features. Recently, various efforts have been made for deep semantic change features, but the importance of grained features has always been ignored, leading to uncertainty of change ground entities. Besides, for an idealistic model, both accuracy and speed need focus. Thus, a novel method EUNet-CD is proposed, based on an efficient nested connection for a robust CD as a remedy for the above issues. The proposed scheme comprises: 1) an efficient convolution module (ECM)-based encoder/decoder to concise the ground entities’ details to minimize the parameters and 2) ultimate fusion strategies (based on spatial/channel attention) act as a repeater to refine the multiscale features and avoid subsampling loss. Experimental validation on different criteria shows the superiority of the proposed EUNet-CD among the state-of-the-arts (SOTA) methods. Our EUNet-CD remarkably improves the overall accuracy and minimizes the computational parameters and time floating point of operations per second (FLOPs) by 33% and 20%.

Infrared (IR) and visible images are heterogeneous data, and their fusion is one of the important research contents in the remote sensing field. In the last decade, deep networks have been widely used in image fusion due to their ability to preserve high-level semantic information. However, due to the lower resolution of IR images, deep learning-based methods may not be able to retain the salient features of IR images. In this article, a novel IR and visible image fusion based on IR Features & Multiscale Dense Network (IR-MSDNet) is proposed to preserve the content and key target features from both visible and IR images in the fused image. It comprises an encoder, a multiscale decoder, a traditional processing unit, and a fused unit, and can capture incredibly rich background details in visible images and prominent target details in IR features. When the dense and multiscale features are fused, the background details are obtained by utilizing attention strategy, and then combined with complimentary edge features. While IR features are extracted by traditional quadtree decomposition and Bezier interpolation, and further intensified by refinement. Finally, both the decoded multiscale features and IR features are used to reconstruct the final fused image. Experimental evaluation with other state-of-the-art fusion methods validates the superiority of our proposed IR-MSDNet in both subjective and objective evaluation metrics. Additional objective evaluation conducted on the object detection (OD) task further verifies that the proposed IR-MSDNet has greatly enhanced the details in the fused images, which bring the best OD results.

During the course of ubiquitous data monitoring in the underwater environment, achieving sustainable communication links among the sensor nodes with astute link quality seems an ordeal challenge. Energy utilization has a direct impact because all active devices are battery dependent and no charging or replacement actions can be made when cost- effective data packet delivery has been set as the benchmark. Hop link inspection and the selection of a Shrewd link through a resurrecting link factor have been nothing short of a bleak challenge, and only possible after meticulous research to develop a shrewd underwater routing synergy using extra porous energy shells (SURS-PES) which has never been conducted before. After broadcasting packets, the sensor node conducts a link inspection phase, thereby, if any link is found to be less than or equal to 50% shaky, the destination receiving node adds its residual energy status and returns it to the source node which adds some unusable energy porous shell to strengthen the link from 5% to a maximum of 90% and sends it only to the targeted node, therefore, an unaltered data packet delivery is anticipated. Performance evaluation was carried out using an NS2 simulator and the obtained results were compared with depth-based routing (DBR) and energy efficient DBR (EEDBR) to observe the outcomes with results that confirmed the previously mentioned direction for research in this area.

Remote sensing image scene classification has drawn significant attention for its potential applications in the economy and livelihoods. Unlike the traditional handcrafted features, the convolutional neural networks (CNNs) provides an excellent avenue in obtaining powerful discriminative features. Although tremendous efforts have been made so far in this domain, there are still many open challenges in scene classification due to the scene complexity with higher within-class diversity and between-class similarity. To solve the above-mentioned problems, D- CapsNet is proposed to learn the richer and more robust features for scene classification. It is an end to end network with four types of layers and incorporates visual attention mechanisms. Its diverse capsules encode different properties of complex image scenes, including deep high-level features, spatial attention based on the fusion of multilayers features, both spatial and channel attention based on high-level features, and their fusion. Experiments on three image scene datasets demonstrate that D-CapsNet outperforms other baselines and state-of-the-art methods with a significant improvement in both classification accuracy and speed.

Zero-shot temporal action detection (ZS-TAD), aiming to recognize and detect new and unseen video actions, is an emerging and challenging task with limited solutions. Recent studies have adapted the vision-language pre-trained model CLIP for this task in a parameter-efficient fine-tuning fashion to achieve open-vocabulary detection. However, they suffer from insufficient vision-text alignment because of the dual-stream structure of CLIP and yield inferior TAD results due to the lack of accurate action prior. In this paper, we target the above limitations and propose to learn multimodal Prompts and Text-Enhanced Actionness (mProTEA) for ZS-TAD. Specifically, we insert learnable layer-wise prompts into the vision and text branches of the frozen CLIP and establish a strong coupling between them, resulting in multimodal prompts that can boost cross-modal alignment. To ease computation costs, we propose to conduct multimodal prompt learning on an image recognition dataset with rich concepts (e.g., ImageNet) first and then keep them frozen during TAD fine-tuning. For improving TAD, we introduce text-enhanced actionness modeling, where we leverage the concise semantics of text to assist the calculation of class-agnostic actionness scores, to offer accurate prior information for both action classification and localization. With the above designs, our mProTEA excels in extensive TAD experiments, surpassing the strong competitor STALE by 5.1% on ActivityNet under the zero-shot setting and achieving state-of-the-art performance in conventional supervised scenarios. Ablation studies confirm the effectiveness of our proposals and show superior domain generalization of multimodal prompts learned on ImageNet against the other 10 image recognition datasets.