Action recognition plays a fundamental role in computer vision and video analysis. Nevertheless, extracting effective spatial-temporal features remains a challenging task. This paper proposes three simple but effective weakly Semantic Guided Modules (SGMs) for both environment constrained and cross domain action recognition. The SGMs are composed of totally 3D convolution and element-wise gated operations, thus they are efficient and easy to be implemented. The semantic guidance is obtained in a weakly supervised manner, in which each video clip is labeled with only an action class instead of pixel level semantics. Benefitting from the semantic guidance, the network (named SGN, abbreviation for Semantic Guided Network) can focus on the salient parts of the video clips. Consequently, the redundant information can be reduced and the model is more robust to noises. Besides, benefitting from the intrinsic property of SGMs, SGN is totally end-to-end trainable. Quantities of experiments on both environment constrained (e.g., Penn, HMDB-51 and UCF-101) and cross domain (e.g., ODAR) action recognition datasets demonstrate its effectiveness. Specifically, SGN gets improvements of 3:7%, 2:1% and 5:2% for Penn, HMDB-51 and UCF-101 than the baseline ResNet3D, respectively, and SGN ranked the 3rd place in ODAR 2017 challenge.

Autoencoder, which learns latent representations of samples in an unsupervised manner, has great potential in computer vision and signal processing. However, the diversity of samples makes learning a component Autoencoder remaining a challenging task. This paper proposes a novel Self-Paced AutoEncoder (SPAE) for unsupervised feature extraction. The motivation behind this work is to take samples gradually from simple to complex into consideration during training, which is similar to the mechanism of knowledge acquisition for humans. Under the unsupervised learning framework constructed on the AutoEncoder infrastructure, our SPAE first learns a weak AutoEncoder via samples with small losses, and then elevates itself to a relatively strong AutoEncoder through samples with large losses. Then SPAE is generalized to temporal domain, resulting to Temporal Self-Paced Autoencoder (TSPAE), where the temporal information is explored and exploited to improve the performance. Typically, TSPAE is capable of compressing temporal sequences into temporal-independent data. Experiments on image classification and action recognition demonstrate the effectiveness of SPAE and TSPAE.

Recognizing the current weather conditions from a single image is of great theoretical significance. It also has potential practical value for daily life and traffic scheduling. To achieve that, typical weather recognition methods focus on learning a general weather description, e.g., sunny, cloudy, foggy, rainy and snowy etc, for the overall weather condition. However, it is far away from being sufficient for many tasks especially traffic management and control. To solve this key problem, this paper proposes a Global-Similarity Local-Salience Network (abbreviated as GSLSNet) for traffic weather recognition. Specifically, a simple but effective Global-Similarity Module (GSM) is proposed to recognize the overall weather condition and a Local-Salience Module (LSM) is presented to restrict the network to focus on road weather details. Besides, this paper also provides a new traffic weather dataset, named TWData, which is the first fine categorized dataset especially for highway weather recognition. Experimental results compared with state-of-the-art methods on both public datasets and TWData demonstrate the superiority of the proposed GSLSNet.

High-resolution wind analysis plays an essential role in pollutant dispersion and renewable energy utilization. This paper focuses on spatial wind downscaling. Specifically, a novel t erra i n g uided fl a tten m emory network (abbreviated as TIGAM) with axial similarity constraint is proposed. TIGAM consists of three elaborately designed blocks, i.e., the similarity block, the reconstruction block, and the denoise block. To achieve long-spatial dependency, the similarity block interpolates low resolution data to high resolution in an axial attention manner. Meanwhile, the reconstruction block aims to obtain a clearer high resolution representation in closed form. Taking both of the meteorological prior and network design principle into consideration, this paper also proposes a flatten memory module with learnable input for high resolution denoising. Furthermore, for accurate detail reconstruction, a terrain guided enhanced loss is presented benefitting from the high-resolution remote sensing data. This loss function integrates wind spatial distribution and terrain elegantly. Extensive quantitative and qualitative experiments demonstrate the superiority of the proposed TIGAM.

Weather forecasting, which is challenging due to the complex atmospheric correlation, focuses on providing explicit meteorological estimations as accurate as possible. Recently, techniques based upon convolutional-recurrent networks have shown dramatic performance in domains including radar echo prediction and precipitation forecasting, indicating that deep learning models have great potential in this area. However, existing methods concentrate on adding extra paralleled memory cells to the inner recurrent unit, where the information is mutually independent. To extract spatial–temporal features with stronger correlation, this letter introduced an axial attention memory module with quasi state-in-state cascaded manner. Benefitting from this unit, the spatial–temporal features can be aggregated and embedded into standard ConvGRU formulating Axial aTtention Memory Cascaded ConvGRU (ATMConvGRU) for sequential weather prediction. Experimental results compared with many state-of-the-art methods on four types of weather forecasting related datasets, including temperature, relative humidity, wind, and radar echo, demonstrate the effectiveness and superiority of the proposed ATMConvGRU.

Precipitation downscaling, which is similar to the mechanism of single-image super-resolution (SR), aims to improve the spatial resolution of rain maps. It is of great practical value and theoretical significance. This letter presents a new deep precipitation downscaling (DPD) method, named auxiliary guided spatial distortion (AGSD) network, motivated by SR techniques. Specifically, an auxiliary guided module (AGM), which takes multiple meteorological elements (e.g., temperature, relative humidity, and wind) as input, is proposed for getting more accurate rain map features. Meanwhile, a simple but effective spatial distortion module (SDM) is proposed. Benefitting from SDM, the DPD model can rectify the rain map via terrain correlation. Furthermore, to improve the model performance among various rain intensity (including small rain, moderate rain, heavy rain, and storm), a threat score-driven pseudo threat-score (PTS) loss is presented. Experimental results compared with state-of-the-art methods demonstrate the superiority of the proposed method.