Estimating 3D human mesh is appealing for various application scenarios. Current mainstream solution predicts the meshes either from the image or the human reflected RF-signals. In this paper, instead of investigating which approach is better, we propose to design a multi-modality fusion framework, namely MI-Mesh, which estimates 3D meshes by fusing image and mmWave. To realize this, we design a deep neural network model. It first automatically correlate mmWave point clouds to certain human joints and extracts useful fused features from two modalities. Then, the features are refined by predicting 2D joints and silhouette. Finally, we regress pose and shape parameters and feed them to SMPL model to generate the 3D human meshes. We build a prototype on commercial mmWave radar and camera. The experimental results demonstrate that with the integration of multi-modality strengths, MI-Mesh can effectively recover human meshes on dynamic motions and across different conditions.

Falls pose a significant risk to the health and well-being of older adults, driving the development of various fall detection systems. Existing solutions have explored wearable and vision sensors, while non-invasive RF-based approaches have raised a growing interest due to their convenience and privacy considerations. Despite major advancements in RF-based passive estimation, current approaches still face challenges in handling complex real-world scenarios. They often lack the ability to generalize to new domains (i.e., people, position, environment), and struggle to accurately detect and localize a fallen person in the presence of unknown activities from nearby objects (e.g., pet animal and robot vacuum cleaner) or persons. To address these challenges, we present mm-Fall, a novel mmWave-based non-invasive fall detection system that utilizes Range-Angle (RA) energy maps to separate and localize multiple moving targets, and further accurately estimate their states. Unlike previous approaches, mm-Fall is capable of working with new domains and effectively distinguishing falls from non-fall motions that may appear similar. Additionally, it performs well in challenging conditions, such as poor lighting and occluded scenarios. Our design of mm-Fall is evaluated in 13 environments with over 16 individuals performing 24+ types of motions. The results demonstrate an impressive average recall of 0.969 and precision of 0.996 in detecting falls, whether involving single or multiple moving targets simultaneously. The source codes and dataset of mmFall are available at https://github.com/iwantlatiao/mmFall.

Radio Frequency Identification (RFID) is widely recognized for its cost-effectiveness, energy efficiency, and ease of manufacturing, promising versatile applications such as material recognition, vibration detection, and localization. However, these inherent lightweight characteristics necessitate extensive computations or tailored devices for specific tasks. Consequently, there is a critical imperative to explore and augment the sensing and communication capabilities of standard RFIDs to better accommodate a broader spectrum of Internet of Things (IoT) services. In this paper, we introduce SENSO, a self-enhancement solution designed to facilitate diverse tasks without the need for hardware modifications or additional device support. Our approach introduces two key innovations to enable cross-protocol communication (RFID-LoRa) and localization. Firstly, we seamlessly integrate Chirp Spread Spectrum (CSS) modulation into RFID, enabling simultaneous communication with commercial tags and LoRa devices. Secondly, we realize the transmission of wider-band chirps to activate and localize the tag concurrently. By harnessing these techniques, we developed a prototype while addressing several challenges. Our evaluation demonstrates that when maintaining the fundamental RFID communication, SENSO can effectively deliver additional messages to LoRa devices across various indoor scenarios, e.g., through multiple floors, rooms, and corners, and extending over outdoor distances spanning kilometers. Furthermore, it achieves an average accuracy of approximately 5 to 10 centimeters in tag ranging and localization. The source codes of SENSO are available at https://github.com/Cui-Zhao/SENSO.

This paper introduces a pioneering framework for music representation learning, which harnesses knowledge graph embeddings to enrich genre classification. Leveraging metadata from publicly available datasets like FMA and OpenMIC-2018, the constructed knowledge graph delineates intricate relationships among genres, artists, and instruments, offering valuable insights for genre representation. Within this framework, we propose two models tailored for distinct genre classification scenarios: fixed-set genre classification and open-set genre classification. These models exploit the knowledge graph to unveil correlations among different genres and integrate this knowledge into the audio representation. Notably, our approach is the first to merge audio data with high-level knowledge for music genre classification. Experimental results demonstrate that our proposed methods outperform state-of-the-art approaches, achieving an average genre classification accuracy of 68.07% on the FMA-medium dataset and 42.4% for open-set classification on the FMA-large dataset.

The ubiquity of illumination facilities enables the versatile development of Visible Light Communication (VLC). VLC-based research achieved high-speed wireless access and decimeter-level indoor localization with complex equipment. However, it is still unclear whether the VLC is applicable for widely-used battery-free Internet-of-Things nodes, e.g., passive RFIDs. This paper proposes LightSign, the first cross-technology system that enables passive RFID tags to receive visible light messages. LightSign is compatible with commercial protocols, transparent to routine RFID communications, and invisible to human eyes. We propose a pseudo-timing instruction to achieve microsecond-level light switching to modulate the VLC message. To make it perceptible to passive RFIDs, we design an augmented RFID tag and prove its effectiveness theoretically and experimentally. With only one reply from an augmented tag, LightSign can decode 100-bit-long VLC messages. We evaluate LightSign in real industry environments and test its performance with two use cases. The results show that LightSign achieves up to 99.2% decoding accuracy in varying scenarios.

Most Personalized Federated Learning (PFL) algorithms merge the model parameters of each client with other (similar or generic) model parameters to optimize the personalized model (PM). However, the merged model parameters in these algorithms may fit low relevance data, thereby limiting the performance of PM. In this paper, we generate similar data for each client through the collaboration of a generic model (GM) on the server, rather than merging model parameters. To train a generator capable of generating data for all classes on the server without real data, we employ the GM as the discriminator in adversarial training with the generator. Additionally, we introduce a similarity assessment metric, which allows for the assessment of the similarity between local data and data from other classes. Nevertheless, the presence of non-IID data among clients can weaken the performance of the GM, consequently impacting the training of the generator and similarity assessment. To address this issue, we design a directive mechanism so that GM can be optimized during adversarial training without the need for additional training. The experimental results validate the superiority of our algorithm over state-of-the-art algorithms in terms of accuracy, loss, and convergence speed.