Recently, dense video captioning has made attractive progress in detecting and captioning all events in a long untrimmed video. Despite promising results were achieved, most existing methods do not sufficiently explore the fine scene evolution within an event temporal proposal for captioning, and therefore perform less satisfactorily when the scenes and objects change over a relatively long proposal. To address this problem, we propose a graph-based partition-and-summarization (GPaS) framework for dense video captioning. By “partition”, a whole event proposal is split into short video segments for captioning at a finer level. By “summarization”, the generated sentences carrying rich description information for each segment are summarized into one sentence to describe the whole event. We particularly focus on the process of sentences summarization, and propose a framework that effectively exploits the relationship between semantic words for summarization. We achieve this goal by treating semantic words as nodes in a graph and learn their interactions by coupling Graph Convolutional Network (GCN) and Long Short Term Memory (LSTM), with the aid of visual cues. Two schemes of GCN-LSTM Interaction (GLI) modules are proposed for seamless integration of GCN and LSTM. The effectiveness of our approach is demonstrated via an extensive comparison with the state-of-the-arts methods on the benchmark ActivityNet Captions dataset.

In this work, we propose a division-and-summarization (DaS) framework for dense video captioning. After partitioning each untrimmed long video as multiple event proposals, where each event proposal consists of a set of short video segments, we extract visual feature (e.g., C3D feature) from each segment and use the existing image/video captioning approach to generate one sentence description for this segment. Considering that the generated sentences contain rich semantic descriptions about the whole event proposal, we formulate the dense video captioning task as a visual cue aided sentence summarization problem and propose a new two stage Long Short Term Memory (LSTM) approach equipped with a new hierarchical attention mechanism to summarize all generated sentences as one descriptive sentence with the aid of visual features. Specifically, the first-stage LSTM network takes all semantic words from the generated sentences and the visual features from all segments within one event proposal as the input, and acts as the encoder to effectively summarize both semantic and visual information related to this event proposal. The second-stage LSTM network takes the output from the first-stage LSTM network and the visual features from all video segments within one event proposal as the input, and acts as the decoder to generate one descriptive sentence for this event proposal. Our comprehensive experiments on the ActivityNet Captions dataset demonstrate the effectiveness of our newly proposed DaS framework for dense video captioning.

In the point cloud analysis task, the existing local feature aggregation descriptors (LFAD) do not fully utilize the neighborhood information of center points. Previous methods only use the distance information to constrain the local aggregation process, which is easy to be affected by abnormal points and cannot adequately fit the original geometry of the point cloud. This paper argues that fine-grained geometric information (FGGI) plays an important role in the aggregation of local features. Based on this, we propose a gradient-based local attention module to address the above problem, which is called Gradient Attention Module (GAM). GAM simplifies the process of extracting the gradient information in the neighborhood to explicit representation using the Zenith Angle matrix and Azimuth Angle matrix, which makes the module 35X faster. The comprehensive experiments on the ScanObjectNN dataset, ShapeNet dataset, S3DIS dataset, Modelnet40 dataset, and KITTI dataset demonstrate the effectiveness, efficientness, and generalization of our newly proposed GAM for 3D point cloud analysis. Especially in S3DIS, GAM achieves the highest index in the current point-based model with mIoU/OA/mAcc of 74.4%/90.6%/83.2%.

Phase-sensitive optical time domain reflectometry (Φ-OTDR) enhances diagnostic capabilities by providing detailed event classification essential for fiber infrastructure management, making it a critical tool in measurement science for real-time monitoring and fault detection in large-scale industrial and environmental systems. Recent deep learning techniques, including convolutional neural networks and long short-term memory networks, have shown promise for this task, though current supervised methods struggle to utilize vast unlabeled data and fail to effectively represent Φ-OTDR signals. To address these issues, we introduce the Φ-masked autoencoder (Φ-MAE) framework, which leverages MAE architecture in two stages: a pre-training phase using unlabeled data to extract robust representations and a fine-tuning phase for classification refinement with labeled data. Extending this, we propose the Φ-global-local MAE (Φ-GLMAE) framework, which integrates local and global patterns for improved feature extraction. Experimental evaluations on the BJTU-OTDR dataset demonstrate that our proposed Φ-GLMAE achieves an accuracy of 99.74 % , outperforming state-of-the-art approaches. In addition, under the 2 % labeled data condition, the proposed method with pre-training achieves 87.36 % accuracy, significantly outperforming the method without pre-training (64.90 % ) by 22.46 % . The source code will be made available at https://github.com/NBT-AILAB/global_local_mae.