We address the challenging problem of weakly supervised temporal action localization from unconstrained web videos, where only the video-level action labels are available during training. Inspired by the Adversarial Erasing strategy in weakly supervised semantic segmentation, we propose a novel iterative-winners-out network. Specifically, we make two technical contributions: 1) we propose an iterative training strategy, namely winners-out, to select the most discriminative action instances in each training iteration and remove them in the next training iteration. This iterative process alleviates the “winner-takes-all” phenomenon that existing approaches tend to choose the video segments that strongly correspond to the video label, but neglect other less discriminative video segments. With this strategy, our network is able to localize not only the most discriminative instances but also the less discriminative ones. 2) to better select the target action instances in winners-out, we devise a class-discriminative localization technique. By employing the attention mechanism and the information learned from data, our technique is able to identify the most discriminative action instances effectively. The two key components are integrated into an end-to-end network to localize actions without using the frame-level annotations. Extensive experimental results demonstrate that our method outperforms the state-of-the-art weakly supervised approaches on ActivityNet1.3 and improves mAP from 16.9% to 20.5% on THUMOS14. Notably, even with weak video-level supervision, our method attains comparable accuracy to those employing frame-level supervisions.

Temporal action localization, which aims at recognizing the location and the category of action instances in videos, has long been researched. Existing methods divide each video into multiple action units (i.e., proposals in two-stage methods and segments in one-stage methods) and then perform recognition/regression on each of them individually without explicitly exploiting their relations, which, however, play an important role in action localization. In this paper, we propose a general graph convolutional module (GCM) that can be easily plugged into existing action localization methods, including two-stage and one-stage paradigms. Specifically, we first construct a graph, where each action unit is represented as a node and their relations as edges. We use two types of relations, one for capturing the temporal connections, and the other one for characterizing the semantic relationship. Then, we apply graph convolutional networks (GCNs) on the graph to model the relations and learn more informative representations for action localization. Experimental results show that GCM consistently improves the performance of both two-stage action localization methods (e.g., CBR and R-C3D) and one-stage methods (e.g., D-SSAD), verifying the generality and effectiveness of GCM. Moreover, with the aid of GCM, our approach significantly outperforms the state-of-the-art on THUMOS14 and ActivityNet.

Geometric graphs are a special kind of graph with geometric features, which are vital to model many scientific problems. Unlike generic graphs, geometric graphs often exhibit physical symmetries of translations, rotations, and reflections, making them ineffectively processed by current Graph Neural Networks (GNNs). To address this issue, researchers proposed a variety of geometric GNNs equipped with invariant/equivariant properties to better characterize the geometry and topology of geometric graphs. Given the current progress in this field, it is imperative to conduct a comprehensive survey of data structures, models, and applications related to geometric GNNs. In this paper, based on the necessary but concise mathematical preliminaries, we formalize geometric graph as the data structure, on top of which we provide a unified view of existing models from the geometric message passing perspective. Additionally, we summarize the applications as well as the related datasets to facilitate later research for methodology development and experimental evaluation. We also discuss the challenges and future potential directions of geometric GNNs at the end of this survey.

Temporal action localization aims to accurately localize and recognize all possible action instances from an untrimmed video automatically. Most existing methods tackle this task by first generating a set of proposals and then recognizing each one independently. However, due to the complex structures and large content variations in action instances, recognizing them one by one can be difficult. Fortunately, some proposals often have shared information regarding one specific action. Such information, which is ignored in existing methods, can be used to boost the recognition performance. In this paper, we propose a novel mechanism, called relation attention, to exploit the informative relations among proposals according to their appearance or optical flow features. Specifically, we propose a relation attention module to enhance the representation power by capturing useful information from other proposals. This module does not change the dimensions of the original input and output and does not rely on any specific proposal generation methods or feature extraction backbone networks. Extensive experimental results show that the proposed relation attention mechanism is able to improve performance significantly on both Thumos14 and ActivityNet1.3 datasets upon several existing best-performing architectures. For example, relying on Structured Segment Networks (SSN), the proposed relation attention module helps to increase thejavascript:setNextPage(‘AUTHOR_VIEW_RESPOND_TO_COMMENTS’) mAP from 41.4 to 43.7 on the Thumos14 dataset and outperform the state-of-the-art results.

As well-known, deep learning models have achieved great success in the field of intelligent fault diagnosis. However, once the working condition changed, the diagnostic accuracy of the trained models would be greatly affected, which seriously limits the application of deep learning models in real industry. Therefore, signal-transformer (S-Transformer), an intelligent fault diagnosis model focusing on the problem of variable operating conditions, is proposed in this study. First, signal embedding is employed to complete the segmentation and up-dimensional representation of the 1-D signal, thus enriching the information in a high dimensional space. Then, those embedded subsignals are further processed via the multihead self-attention mechanism to explore the state features of the signal in different spaces for a deep representation. Finally, an attention visualization method is proposed for vibrating signals to increase the interpretability of the proposed model and overcome the drawbacks of black boxes. With an experimental validation, the proposed model outperforms than other six deep learning methods in terms of diagnostic accuracy under unknown operating conditions. According to the principle of the model, the crucial weights for signal are further explained in visualization. Furthermore, it can be foreseen that the S-Transformer can achieve a robust recognition effect with the ability of interpretable feature extraction in signal denoising.

This paper proposes a rotational energy harvester using a piezoelectric bistable buckled beam to harvest low-speed rotational energy. The proposed harvester consists of a piezoelectric buckled beam with a center magnet, and a rotary magnet pair with opposite magnetic poles mounted on a revolving host. The magnetic plucking is used to harvest the angular kinetic energy of the host. The nonlinear snap-through mechanism is utilized to improve the vibration displacement and output voltage of the piezoelectric layer over a wide rotation frequency range. Theoretical simulation and experimental results show that the proposed energy harvester can yield a stable average output power ranging between 6.91-48.01 μW over a rotation frequency range of 1-14 Hz across a resistance load of 110 kΩ. Furthermore, dual attraction magnets were employed to overcome the suppression phenomenon at higher frequencies, which yields a broadband and flat frequency response over 6-14 Hz with the output power reaching 42.19-65.44 μW, demonstrating the great potential of the bistable buckled beam for wideband rotation motion energy harvesting.