Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.
Existing vision-based fatigue detection methods commonly utilize RGB cameras to extract facial and physiological features for monitoring driver fatigue. These features often include single indicators such as eyelid movement, yawning frequency, and heart rate. However, the accuracy of RGB cameras can be affected by factors like varying lighting conditions and motion. To address these challenges, we propose a non-invasive method for multi-modal fusion fatigue detection called RPPMT-CNN-BiLSTM. This method incorporates a feature extraction enhancement module based on the improved Pan–Tompkins algorithm and 1D-MTCNN. This enhances the accuracy of heart rate signal extraction and eyelid features. Furthermore, we use one-dimensional neural networks to construct two models based on heart rate and PERCLOS values, forming a fatigue detection model. To enhance the robustness and accuracy of fatigue detection, the trained model data results are input into the BiLSTM network. This generates a time-fitting relationship between the data extracted from the CNN, allowing for effective dynamic modeling and achieving multi-modal fusion fatigue detection. Numerous experiments validate the effectiveness of the proposed method, achieving an accuracy of 98.2% on the self-made MDAD (Multi-Modal Driver Alertness Dataset). This underscores the feasibility of the algorithm. In comparison with traditional methods, our approach demonstrates higher accuracy and positively contributes to maintaining traffic safety, thereby advancing the field of smart transportation.
Accidents occur in the construction industry as a result of non-compliance with personal protective equipment (PPE). As a result of diverse environments, it is difficult to detect PPE automatically. Traditional image detection models like convolutional neural network (CNN) and vision transformer (ViT) struggle to capture both local and global features in construction safety. This study introduces a new approach for automating the detection of personal protective equipment (PPE) in the construction industry, called PPE_Swin. By combining global and local feature extraction using the self-attention mechanism based on Swin-Unet, we address challenges related to accurate segmentation, robustness to image variations, and generalization across different environments. In order to train and evaluate our system, we have compiled a new dataset, which provides more reliable and accurate detection of personal protective equipment (PPE) in diverse construction scenarios. Our approach achieves a remarkable 97% accuracy in detecting workers with and without PPE, surpassing existing state-of-the-art methods. This research presents an effective solution for enhancing worker safety on construction sites by automating PPE compliance detection.
To address challenges in stock price prediction including data nonlinearity and anomalies, we propose a hybrid CNN-BiGRU-AM framework integrated with deep learning-based anomaly detection. First, an anomaly detection module identifies irregularities in stock price data. The CNN component then extracts local features while filtering anomalous information, followed by nonlinear pattern modeling through BiGRU with attention mechanisms. Final predictions undergo secondary anomaly screening to ensure reliability. Experimental evaluation on Shanghai Composite (SSE) daily closing prices demonstrates superior performance with = 0.9903, RMSE = 22.027, MAE = 19.043, and a Sharpe Ratio of 0.65. It is noteworthy that the MAE of this model is reduced by 14.7%, and the RMSE is decreased by 7.7% compared to its ablation model. The framework achieves multi-level feature extraction through convolutional operations and bidirectional temporal modeling, effectively enhancing model generalization via nonlinear mapping and anomaly correction. Comparative Sharpe Ratio analysis across models provides practical insights for investment decision-making. This dual-functional system not only improves prediction accuracy but also offers interpretable references for market mechanism analysis and regulatory policy formulation.
The precise extraction of displacement time series for complex landslides poses significant challenges, and conventional landslide prediction models often overlook the deformation impacts of displacement triggers. To address this, we introduce a novel composite model tailored for predicting landslide displacement. This model employs Variational Mode Decomposition (VMD) to isolate each displacement component, with optimization achieved through the groupwise coupling algorithm. Subsequently, Grey correlation analysis (GRA) is applied to quantitatively assess the dynamic correlations between various triggering factors and landslide displacement. This analysis informs the construction of a feature set predicated on these correlation factors. Integrating the time-series VMD module into the standard Transformer architecture facilitates the prediction of landslide displacement. This integration allows for the extraction of critical time-evolution features associated with the displacement components. Ultimately, the predicted displacements are aggregated and reconstructed. We validate our model using the Bazimen landslide case study, analyzing displacement monitoring data from 1 January 2007, to 31 December 2012. The values of the root mean square error and the mean absolute percentage error were 1.86 and 4.85, respectively. This model offers a more nuanced understanding of the multifaceted causes and evolutionary dynamics underpinning landslide displacement and deformation, thereby markedly enhancing prediction accuracy.
As small commodity features are often few in number and easily occluded by hands, the overall detection accuracy is low, and small commodity detection is still a great challenge. Therefore, in this study, a new algorithm for occlusion detection is proposed. Firstly, a super-resolution algorithm with an outline feature extraction module is used to process the input video frames to restore high-frequency details, such as the contours and textures of the commodities. Next, residual dense networks are used for feature extraction, and the network is guided to extract commodity feature information under the effects of an attention mechanism. As small commodity features are easily ignored by the network, a new local adaptive feature enhancement module is designed to enhance the regional commodity features in the shallow feature map to enhance the expression of the small commodity feature information. Finally, a small commodity detection box is generated through the regional regression network to complete the small commodity detection task. Compared to RetinaNet, the F1-score improved by 2.6%, and the mean average precision improved by 2.45%. The experimental results reveal that the proposed method can effectively enhance the expressions of the salient features of small commodities and further improve the detection accuracy for small commodities.