In this paper, we consider hybrid parallelism---a paradigm that employs both Data Parallelism (DP) and Model Parallelism (MP)---to scale distributed training of large recommendation models. We propose a compression framework called Dynamic Communication Thresholding (DCT) for communication-efficient hybrid training. DCT filters the entities to be communicated across the network through a simple hard-thresholding function, allowing only the most relevant information to pass through. For communication efficient DP, DCT compresses the parameter gradients sent to the parameter server during model synchronization. The threshold is updated only once every few thousand iterations to reduce the computational overhead of compression. For communication efficient MP, DCT incorporates a novel technique to compress the activations and gradients sent across the network during the forward and backward propagation, respectively. This is done by identifying and updating only the most relevant neurons of the neural network for each training sample in the data. We evaluate DCT on publicly available natural language processing and recommender models and datasets, as well as recommendation systems used in production at Facebook. DCT reduces communication by at least 100x and 20x during DP and MP, respectively. The algorithm has been deployed in production, and it improves end-to-end training time for a state-of-the-art industrial recommender model by 37%, without any loss in performance.

Accurate acquisition for the positions of the waterlines plays a critical role in coastline extraction. However, waterline extraction from high-resolution images is a very challenging task because it is easily influenced by the complex background. To fulfill the task, two types of vision transformers, segmentation transformers (SETR) and semantic segmentation transformers (SegFormer), are introduced as an early exploration of the potential of transformers for waterline extraction. To estimate the effects of the two methods, we collect the high-resolution images from the web map services, and the annotations are created manually for training and test. Through extensive experiments, transformer-based approaches achieved state-of-the-art performances for waterline extraction in the artificial coast.

As high-resolution remote sensing images begin to integrate new characteristics, such as a great volume of data, a wide variety of ground objects, and high structural complexity, traditional methods previously used for feature extraction in low-resolution remote sensing images are inefficient and inadequate for the accurate feature description of various objects. Thus, object feature extraction from a high-resolution remote sensing image remains a challenging task. To address this issue, we introduced the visual attention mechanism into high-resolution remote sensing image analysis in this study by proposing a novel object-oriented random walk model for visual saliency (ORWVS) detection from high-resolution remote sensing images. In the proposed model, an object-oriented random walk strategy is designed to simulate the transfer path of visual focus on the images and to extract the local salient regions in an efficient and accurate manner, laying a foundation for accurate feature descriptors. The ORWVS is compared with eight visual attention models, and the experiments prove its superiority.

Underwater image enhancement is critical for a variety of marine applications such as exploration, navigation, and biological research. However, underwater images often suffer from quality degradation due to factors such as light absorption, scattering, and color distortion. Although current deep learning methods have achieved better performance, it is difficult to balance the enhancement performance and computational efficiency in practical applications, and some methods tend to cause performance degradation on high-resolution large-size input images. To alleviate the above points, this paper proposes an efficient network GFRENet for underwater image enhancement utilizing gated linear units (GLUs) and fast Fourier convolution (FFC). GLUs help to selectively retain the most relevant features, thus improving the overall enhancement performance. FFC enables efficient and robust frequency domain processing to effectively address the unique challenges posed by the underwater environment. Extensive experiments on benchmark datasets show that our approach significantly outperforms existing state-of-the-art techniques in both qualitative and quantitative metrics. The proposed network provides a promising solution for real-time underwater image enhancement, making it suitable for practical deployment in various underwater applications.

The road network is of great importance to the modern traffic system, which changes frequently as its high speed of development. Simultaneously, a growing number of satellites provide a considerable number of remote sensing images that can be used for road extraction. However, most existing methods cannot get accurate results in the high-resolution remote sensing, since the complex background like the shadows and occlusions. Consequently, we proposed a method utilizing the volunteered geographic information data to extract the road from satellite images. At first, from Mapbox platform, the Volunteered Geographic Information data, OpenStreetMap vector tiles are collected to guide the downloading of corresponding road satellite images. Then a level set method is used to extract the road areas from the satellite images supported by the road vector. After that, a superpixel method is applied to improve the road areas by fusing the superpixels with the previous results. Experiments are performed on the satellite images from Mapbox, Google, and Yahoo in the Tianjin Port district on accuracy and efficiency. Experimental results demonstrate that our approach is more accurate and of higher performance than the other two state-of-the-art methods.

Dehazing individual remote sensing (RS) images is an effective approach to enhance the quality of hazy remote sensing imagery. However, current dehazing methods exhibit substantial systemic and computational complexity. Such complexity not only hampers the straightforward analysis and comparison of these methods but also undermines their practical effectiveness on actual data, attributed to the overtraining and overfitting of model parameters. To mitigate these issues, we introduce a novel dehazing network for non-uniformly hazy RS images: GLUENet, designed for both lightweightness and computational efficiency. Our approach commences with the implementation of the classical U-Net, integrated with both local and global residuals, establishing a robust base for the extraction of multi-scale information. Subsequently, we construct basic convolutional blocks using gated linear units and efficient channel attention, incorporating depth-separable convolutional layers to efficiently aggregate spatial information and transform features. Additionally, we introduce a fusion block based on efficient channel attention, facilitating the fusion of information from different stages in both encoding and decoding to enhance the recovery of texture details. GLUENet’s efficacy was evaluated using both synthetic and real remote sensing dehazing datasets, providing a comprehensive assessment of its performance. The experimental results demonstrate that GLUENet’s performance is on par with state-of-the-art (SOTA) methods and surpasses the SOTA methods on our proposed real remote sensing dataset. Our method on the real remote sensing dehazing dataset has an improvement of 0.31 dB for the PSNR metric and 0.13 for the SSIM metric, and the number of parameters and computations of the model are much lower than the optimal method.