The latest video coding standard, versatile video coding (VVC), was developed to achieve higher video compression efficiency and support more media applications than its predecessor, high-efficiency video coding (HEVC). To address nontranslational motion, such as rotation and zooming, affine motion compensation has been employed in VVC during interframe prediction. However, the complexity increases significantly due to a large number of linear equation solving steps during affine motion estimation (AME). To address the problem, this paper proposes a fast linear equation solving algorithm and an accompanying pipelined hardware architecture design. To the best of our knowledge, our work is the first attempt to address the hardware architecture design of the linear equation solving algorithm in affine mode. First, an integer-based division-free algorithm (I-DFA) is proposed to achieve fast equation solving. Then, a novel dynamic scaling algorithm is proposed to compensate for integer computation errors due to overflow problems. Finally, a pipelined and interleaved hardware architecture is proposed to minimize the number of iteration clock cycles and improve the throughput. The proposed algorithm achieves average time savings of 5.3% and 5.7% with only 0.03% and 0.07% increase in the Bjøntegaard delta bit rate (BD-BR) under low-delay P (LDP) and random access (RA) configurations, respectively. The proposed hardware architecture can solve 16.7M six-parameter affine systems of linear equations per second under a working frequency of 100MHz, which represents a 21x improvement compared to the existing methods.
22 July 2023 in Journal of Real-Time Image Processing
Multi object tracking (MOT) in UAV imagery is challenged by weak feature representation of small objects due to limited resolution, which leads to frequent missed detections. However, enhancing small object features often amplifies background noise and increases false positives. To address this contradiction, we propose the Temporal Aware Small Object Enhancement Network (TASONet), which integrates spatial enhancement and temporal modeling for robust tracking. The Small Object Enhancement (SOE) module combines depthwise separable convolutions with contrast-aware attention mechanisms (SimAM and LCDAttn) to improve local discriminability. It further incorporates the Small Target Enhancement Path (STEP), which uses motion-difference cues and a confidence adaptive suppression strategy to strengthen spatial features while mitigating noise. The Temporal Enhancement Module (TEM), consisting of Temporal Feature Alignment (TFA) and a Target Memory Unit (TMU), aggregates multi-frame information through adaptive inter-frame fusion and memory of high confidence historical features, improving temporal consistency and reducing false positives potentially introduced by SOE. Experiments show that TASONet achieves significant gains over state-of-the-art methods: on UAVDT, MOTA increases from 68.33 to 75.97 and IDF1 from 83.50 to 88.51; on VisDrone-MOT, MOTA rises from 61.15 to 73.52 with an IDF1 of 88.83. These results validate the effectiveness of jointly enhancing spatial features and temporal coherence for UAV small-object MOT.
JPEG images are widely used by individual users, data centers, cloud storages and network file systems. The huge transmission and storage bandwidth of existing JPEG images is becoming a big challenge due to its low compression efficiency. Some methods such as Google’s Brunsli can further compress these JPEG images adequately and restore them back to JPEG format losslessly when needed, but its decoding speed is not fast enough especially for real-time mobile applications. To further reduce its decoding complexity, we proposed three optimizations based on Brunsli, which include using asymmetric numeral systems coding to replace the existing arithmetic coding, designing the joint encoding of multiple symbols and cache-friendly optimizing the data structures. Experimental results demonstrate that compared with Brunsli, the proposed method achieves average 2 to 2.4 times decoding faster on mobile platform with comparable compression efficiency which is really valuable for power sensitive real-time mobile applications.
15 September 2026 in Journal of Real-Time Image Processing