Just Recognizable Distortion (JRD) refers to the minimum distortion that notably affects the recognition performance of a machine vision model. If a distortion added to images or videos falls within this JRD threshold, the degradation of the recognition performance will be unnoticeable. Based on this JRD property, it will be useful to Video Coding for Machine (VCM) to minimize the bit rate while maintaining the recognition performance of compressed images. In this study, we propose a deep learning-based JRD prediction model for image and video compression. We first construct a large image dataset of Object-Wise JRD (OW-JRD) containing 29,218 original images with 80 object categories, and each image was compressed into 64 distorted versions using Versatile Video Coding (VVC). Secondly, we analyze of the distribution of the OW-JRD, formulate JRD prediction as binary classification problems and propose a deep learning-based OW-JRD prediction framework. Thirdly, we propose a deep learning based binary OW-JRD predictor to predict whether an image object is still detectable or not under different compression levels. Also, we propose an error-tolerance strategy that corrects misclassifications from the binary classifier. Finally, extensive experiments on large JRD image datasets demonstrate that the Mean Absolute Errors (MAEs) of the predicted OW-JRD are 4.90 and 5.92 on different numbers of the classes, which is significantly better than the state-of-the-art JRD prediction model. Moreover, ablation studies on deep network structures, object sizes, features, data padding strategies and image/video coding schemes are presented to validate the effectiveness of the proposed JRD model.
With the development of communication networks and the widespread use of mobile terminals, videos are increasingly shared and distributed among mobile users. During the process, the video is recompressed to a certain file size on the sending side and sent to the receiving side via the server. This makes robust video watermark with low complexity to resist recompression attacks become an important issue to address. The proposed video watermarking method in this article is specially designed for resisting recompression attacks when quantization parameter (QP) is greatly increased. In the proposed method, by using the texture information and motion information of video, the invariance of video content under different quantization parameters could be found to help improve the anti-recompression attack ability of video watermark. Moreover, the proposed framework does not use a location map which has security risk to locate the watermark. It aims at finding the optimal location of watermark embedding adaptively according to the feature of video content itself. The experiment results indicate that the proposed video watermarking method can not only achieve greater robustness against recompression attack but also effectively limit the degradation in video perceptual quality.
Urban Functional Zones (UFZs) are spatial units of the city divided according to specific functional activities. Detailed identification of UFZs is vital for optimizing urban management, guiding planning and design, and promoting sustainable development. However, existing UFZ recognition methods face significant challenges, such as difficulties in effectively integrating multi-source heterogeneous data, capturing dynamic spatiotemporal patterns, and addressing the complex interrelationships among various data types. These issues significantly limit the applicability of UFZ mapping in complex urban scenarios. To address these challenges, this paper proposes a tripartite neural network (TriNet) for multimodal data processing, including Remote Sensing (RS) images, Point of Interest (POI) data, and Origin–Destination (OD) data, fully utilizing the complementarity of different data types. TriNet comprises three specialized branches: ImgNet for spatial features extraction from images, POINet for functional density distribution features extraction from POI data, and TrajNet for spatiotemporal pattern features extraction from OD data. Finally, the method deeply fuses these features through a feature fusion module, which utilizes a two-layer fully connected network for deep fusion, allowing the model to fully utilize the interdependencies among the data types, significantly improving the UFZ classification accuracy. The experimental data are generated by mapping OpenStreetMap (OSM) vector into conceptual representations, integrating images with social sensing data to create a comprehensive UFZ classification benchmark. The method achieved an overall accuracy of 84.13% on the test set of Chongqing’s main urban area, demonstrating high accuracy and robustness in UFZ classification tasks. The experimental results show that the TriNet model performs effectively in UFZ classification.
25 August 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
Due to intra-modality variations and cross-modality discrepancy, visible-infrared person re-identification (VI Re-ID) is an important and challenging task in intelligent video surveillance. The cross-modality discrepancy is mainly caused by the differences between visible images and infrared images, the inherent essence of which is heterogeneous. To alleviate this discrepancy, we propose a Dynamic Weighted Gradient Reversal Network (DGRNet) to enhance the learning of discriminative common representations by confusing the modality discrimination. In the proposed DGRNet, we design the gradient reversal model guiding adversarial training between identity classifier and modality discriminator to reduce the modality discrepancy of the same person in different modalities. Furthermore, we propose an optimization training method, that is, designing dynamic weight of gradient reversal to achieve optimal adversarial training, and dynamic weight has the ability to dynamically and adaptively evaluate the significance of target loss term, without involving hyper-parameter tuning. Extensive experiments were conducted on two public VI Re-ID datasets, SYSU-MM01 and RegDB. The experimental results show that the proposed DGRNet outperforms state-of-the-art methods and demonstrate the effectiveness of the DGRNet to learn more discriminative common representations for VI Re-ID.