Human activity recognition (HAR) algorithms today are designed and evaluated on data collected in controlled settings, providing limited insights into their performance in real-world situations with noisy and missing sensor data and natural human activities. We present a real-world HAR open dataset compiled from a wristband equipped with a triaxial accelerometer. During data collection, participants had autonomy in their daily life activities, and the process remained unobserved and uncontrolled. A general convolutional neural network model was trained on this dataset, achieving a mean balanced accuracy (MBA) of 80%. Personalizing the general model through transfer learning can yield comparable and even superior results using fewer data, with the MBA improving to 85%. To emphasize the issue of insufficient real-world training data, we conducted training of the model using the public MHEALTH dataset, resulting in 100% MBA. However, upon evaluating the MHEALTH-trained model on our real-world dataset, the MBA drops to 62%. After personalizing the model with real-world data, an improvement of 17% in the MBA is achieved. This paper showcases the potential of transfer learning to make HAR models trained in different contexts (lab vs. real-world) and on different participants perform well for new individuals with limited real-world labeled data available.
7 December 2020 in IEEE Transactions on Circuits and Systems for Video Technology
Forensic watermarking is used to track down digital pirates after they illegally redistribute video content. Although existing algorithms often resist common signal processing attacks, they are not always robust against camcording attacks. As a solution in the state of the art, registration methods are used to align the attacked video to the original one. However, watermark detection still fails when the quality is sufficiently decreased or when exposed to targeted attacks. Therefore, this paper proposes a novel fallback system that aims to detect the watermark when traditional methods fail. More concretely, we demonstrate that a primary watermark embedded by a traditional scheme indirectly creates a secondary watermark signal during video encoding. This secondary watermark consists of compression artifacts and is detected by the fallback system. Additionally, the proposed system incorporates video registration to cope with camcording attacks. The experimental results indicate that the fallback system has a striking increase in robustness compared to the existing methods. For example, the observed false-negative rate for targeted attacks improves from 100% to 0%. Moreover, the fallback is camcording resistant even when the traditional method combined with registration is not. In conclusion, the proposed system can be used as a fallback when traditional detection fails.
Watermarking enables the identification of digital pirates that illegally redistribute copyrightprotected videos. One of the main challenges is for the watermark to be imperceptible, while not increasing the video bit rate. Additionally, the system should be robust to attacks that attempt to remove the watermark. Therefore, this paper proposes a robust watermarking technique that does not degrade the video quality nor negatively affect the video bit rate. In other words, it preserves the video encoder’s compression efficiency or rate-distortion performance. For watermark embedding, the quantization parameters are varied during video compression. As a result, different compression artifacts are introduced, although they do not distort the video more than those that occur during ordinary video compression. The collection of artifacts represents the watermark and is used for watermark detection. The experimental results prove that the proposed approach retains the rate-distortion performance better than state-of-the-art techniques. Furthermore, the watermarks are robust to recompression and noise attacks. In conclusion, the proposed method enables content providers to perform forensic watermarking without affecting the compression efficiency.
18 October 2023 in Multimedia Tools and Applications
Several frameworks have been proposed for delivering interactive, panoramic, camera-captured, six-degrees-of-freedom video content. However, it remains unclear which framework will meet all requirements the best. In this work, we focus on a Steered Mixture of Experts (SMoE) for 4D planar light fields, which is a kernel-based representation. For SMoE to be viable in interactive light-field experiences, real-time view synthesis is crucial yet unsolved. This paper presents two key contributions: a mathematical derivation of a view-specific, intrinsically 2D model from the original 4D light field model and a GPU graphics pipeline that synthesizes these viewpoints in real time. Configuring the proposed GPU implementation for high accuracy, a frequency of 180 to 290 Hz at a resolution of pixels on an NVIDIA RTX 2080Ti is achieved. Compared to NVIDIA’s instant-ngp Neural Radiance Fields (NeRFs) with the default configuration, our light field rendering technique is 42 to 597 times faster. Additionally, allowing near-imperceptible artifacts in the reconstruction process can further increase speed by 40%. A first-order Taylor approximation causes imperfect views with peak signal-to-noise ratio (PSNR) scores between 45 dB and 63 dB compared to the reference implementation. In conclusion, we present an efficient algorithm for synthesizing 2D views at arbitrary viewpoints from 4D planar light-field SMoE models, enabling real-time, interactive, and high-quality light-field rendering within the SMoE framework.
1 January 2026 in IEEE Transactions on Circuits and Systems for Video Technology