Metaverse technologies are transforming how distant individuals interact in immersive virtual environments. These technologies, in combination with the latest developments in 3D sensing, will create a hybrid metaverse blending virtual and physical spaces. However, high-resolution 3D sensing poses privacy risks from unrestricted spatial data sharing. It may inadvertently expose the human visual appearance and private objects and spaces in the users’ physical environments to the shared metaverse. To mitigate these challenges, this article surveys the visual privacy issues in the era of the metaverse. It then introduces a visual privacy control method enabling privacy protection in the current metaverse and beyond. Our goal is to empower users to control the visual privacy of spatial objects by comprehending their inherent semantics. The design allows users to define their privacy protection levels or, if preferred, delegate this responsibility to an automated control system. We present two use-case scenarios to demonstrate this concept.

Social isolation is an objective absence or paucity of contacts and interactions between a person and a social network. Loneliness is a subjective feeling of being alone, separated, or apart from others. These two highly correlated mental health concerns significantly increase the elderly's risk of premature death from all causes, a risk that may rival those of smoking, obesity, and physical inactivity. In this work, we propose an integrated design for technology usage to tackle the aforementioned mental healthcare concerns involving the elderly community. The technology infrastructure is an Internet-of-Things (IoT) monitoring system aiming to detect social isolation and promptly predict the risk of loneliness. With our system, interventions can be developed to maintain the mental well-being of the elderly, a salient need due to the social disruptions incurred by COVID-19.

Loneliness has long affected the elderly community. This issue is significantly worsened by the social isolation resulting from the COVID-19 pandemic. To address this pressing issue, we employed a sensor-based methodology to predict loneliness and potentially inform interventions. We deployed sensors in the residences of 22 elderly participants from US and Japan, gathering daily activities data through 22 sensor features. Given the extensive feature set, we identify the most effective sensors to ensure unobtrusiveness while upholding privacy. Regression analysis of these features revealed that our best-performing Random Forest model achieved an R2 value of 0.86, on par with existing literature. In addition, we found that the sleep mattress sensor and temperature-humidity sensor were particularly indicative of loneliness. In summary, our research contributes to the HCI literature with effective non-invasive sensing modalities in assessing elderly loneliness, together with insights from our real-world sensor deployments in US and Japan-based elderly communities.

In recent years, school attendance issues among university students have been increasing, which can lead to repeating courses, dropping out of school, or even social withdrawal. Despite the existence of counseling services, students often delay help-seeking, which can cause symptoms to worsen and make support more difficult. Thus, it is essential to identify at-risk students early and encourage them to seek help. A realistic approach must minimize the burden on students, rely only on devices they already own, and operate correctly even for students who are less engaged or prone to social withdrawal. While several techniques have been proposed to estimate individual indicators, they fail to address one of these requirements due to requiring additional devices or requiring user attention and interaction. In this paper, we propose an unobtrusive screening method for detecting subtle signs of school attendance issues in university students. We develop a smartphone app to collect sensor data and collect ground truth information using questionnaires for (1) sleep problems and (2) decreased student engagement. We collect data from 58 university students for about 10 months, and build estimation models for the above indicators. Our evaluation shows that the estimation models are sufficiently accurate in flagging problematic cases. The indicators can then be used to notify at-risk students and medical practitioners, enabling timely intervention. This screening is not intended to replace traditional face-to-face medical examinations, but rather to selectively flag at-risk students and connect them with medical experts.

Prolonged loneliness in older adults increases the risk of dementia, cardiovascular diseases, and premature death. However, timely detection and behavioral pattern analysis are challenging due to the slow progression of loneliness and diverse individual lifestyles. To address these barriers, we propose SMILE, an end-to-end framework for sensor-driven detection and counterfactual pattern analysis of loneliness. Our approach uses non-intrusive sensors to passively observe behavior and build a personalized profile over time. When SMILE detects meaningful changes in activity patterns, it employs generative models to produce counterfactual sensor profiles—synthetic representations of behavior predicted to result in lower loneliness scores. Suggested pattern changes are identified as the difference between observed and counterfactual profiles, highlighting the behaviors most associated with loneliness. We conducted a 6-month feasibility study with 18 participants from the United States and 15 from Japan to develop and evaluate SMILE. Our framework integrates a Time-series Transformer (TST) model for loneliness detection, which achieved 96.25% accuracy, and a diffusion model for generating behavioral explanations, which outperformed a baseline by up to 76.69%. These findings highlight the potential of SMILE to support sensor-driven, personalized loneliness detection and behavioral explanation in aging populations.

Advances in image analysis and deep learning technologies have expanded the use of floor plans, traditionally used for sales and rentals, to include 3D reconstruction and automated design. However, a typical floor plan does not provide detailed information, such as the type and number of outlets and locations affecting the placement of furniture and appliances. Electrical plans, providing details on electrical installations, are intricate due to overlapping symbols and lines and remain unutilized as house manufacturers independently manage them. This paper proposes an analysis method that extracts the house structure, room semantics, connectivities, and specifics of wall and ceiling sockets from electrical plans, achieving robustness to noise and overlaps by leveraging the unique features of symbols and lines. The experiments using 544 electrical plans show that our method achieved better accuracy (+3.6 pt) for recognizing room structures than the state-of-the-art method, 87.2% in identifying room semantics and 97.7% in detecting sockets.