Deep learning (DL)-based Human Activity Recognition (HAR) using wearable inertial measurement unit (IMU) sensors can revolutionize continuous health monitoring and early disease prediction. However, most DL HAR models are untested in their robustness to real-world variability, as they are trained on limited lab-controlled data. In this study, we isolated and analyzed the effects of the subject, device, position, and orientation variabilities on DL HAR models using the HARVAR and REALDISP datasets. The Maximum Mean Discrepancy (MMD) was used to quantify shifts in the data distribution caused by these variabilities, and the relationship between the distribution shifts and model performance was drawn. Our HARVAR results show that different types of variability significantly degraded the DL model performance, with an inverse relationship between the data distribution shifts and performance. The compounding effect of multiple variabilities studied using REALDISP further underscores the challenges of generalizing DL HAR models to real-world conditions. Analyzing these impacts highlights the need for more robust models that generalize effectively to real-world settings. The MMD proved valuable for explaining the performance drops, emphasizing its utility in evaluating distribution shifts in HAR data.

Deep Learning (DL) has enabled considerable increases in the accuracy of classification tasks in several domains, including Human Activity Recognition (HAR). It is well-known that when data distribution changes between the training and test datasets, the accuracy can drop, sometimes significantly. However, some variability sources in HAR, such as sensor orientation, are only sometimes considered when evaluating these models. Therefore, we must understand how much such changes could impact current DL architectures. In this paper, under an orientation variability scenario, we evaluate three common DL architectures, DeepConvLSTM, TinyHAR, and Attend-and-Discriminate, to quantify the performance drop attributed to this shift. Our results show that all architectures show performance drops on average, as expected, but participants are affected differently from them, so they would fall short for some in classification accuracy in real-life settings where orientation can change across the wearing sessions of one participant or across participants. The performance change is related to the difference in distribution distance.