Lala Shakti Swarup Ray

Lala Shakti Swarup Ray

Auto-generated profile, based on publicly available information

Lala Shakti Swarup Ray

DFKI, Kaiserslautern, Germany

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
9 July 2024 in Scientific Reports
Verified
Journal Article
Open access
PDF file
DGDaniel Geißler
Daniel Geißler
BZBo Zhou
Bo Zhou
HBHymalai Bello
Hymalai Bello
JSJoanna Sorysz
Joanna Sorysz
LRLala Ray
Lala Ray
PLPaul Lukowicz
Paul Lukowicz
This work presents a novel and versatile approach to employ textile capacitive sensing as an effective solution for capturing human body movement through fashionable and everyday-life garments. Conductive textile patches are utilized for sensing the movement, working without the need for strain or direct body contact, wherefore the patches can sense only from their deformation within the garment. This principle allows the sensing area to be decoupled from the wearer’s body for improved wearing comfort and more pleasant integration. We demonstrate our technology based on multiple prototypes which have been developed by an interdisciplinary team of electrical engineers, computer scientists, digital artists, and smart fashion designers through several iterations to seamlessly incorporate the technology of capacitive sensing with corresponding design considerations into textile materials. The resulting accumulation of textile capacitive sensing wearables showcases the versatile application possibilities of our technology from single-joint angle measurements towards multi-joint body part tracking.

27 March 2023 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
Bo ZhouBo Zhou
Bo Zhou
DGDaniel Geissler
Daniel Geissler
MFMarc Faulhaber
Marc Faulhaber
CGClara Elisabeth Gleiss
Clara Elisabeth Gleiss
EZEsther Friederike Zahn
Esther Friederike Zahn
Paul LukowiczPaul Lukowicz
Paul Lukowicz
We present MoCaPose, a novel wearable motion capturing (MoCap) approach to continuously track the wearer's upper body's dynamic poses through multi-channel capacitive sensing integrated in fashionable, loose-fitting jackets. Unlike conventional wearable IMU MoCap based on inverse dynamics, MoCaPose decouples the sensor position from the pose system. MoCaPose uses a deep regressor to continuously predict the 3D upper body joints coordinates from 16-channel textile capacitive sensors, unbound by specific applications. The concept is implemented through two prototyping iterations to first solve the technical challenges, then establish the textile integration through fashion-technology co-design towards a design-centric smart garment. A 38-hour dataset of synchronized video and capacitive data from 21 participants was recorded for validation. The motion tracking result was validated on multiple levels from statistics (R2 ~ 0.91) and motion tracking metrics (MP JPE ~ 86mm) to the usability in pose and motion recognition (0.9 F1 for 10-class classification with unsupervised class discovery). The design guidelines impose few technical constraints, allowing the wearable system to be design-centric and usecase-specific. Overall, MoCaPose demonstrates that textile-based capacitive sensing with its unique advantages, can be a promising alternative for wearable motion tracking and other relevant wearable motion recognition applications.

8 October 2023
Verified
Conference Paper
Open access
HBHymalai Bello
Hymalai Bello
Sungho SuhSungho Suh
Sungho Suh
DGDaniel Geißler
Daniel Geißler
LRLala Shakti Swarup Ray
Lala Shakti Swarup Ray
Bo ZhouBo Zhou
Bo Zhou
Paul LukowiczPaul Lukowicz
Paul Lukowicz
We present CaptAinGlove, a textile-based, low-power (≤ 1.15Watts), privacy-conscious, real-time on-the-edge (RTE) glove-based solution with a tiny memory footprint (≤ 2MB), designed to recognize hand gestures used for drone control. We employ lightweight convolutional neural networks as the backbone models and a hierarchical multimodal fusion to reduce power consumption and improve accuracy. The system yields an F1-score of 80% for the offline evaluation of nine classes; eight hand gesture commands and null activity. For the RTE, we obtained an F1-score of 67% (one user).

29 July 2024 in Scientific Reports
Verified
Journal Article
Open access
PDF file
LRLala Ray
Lala Ray
DGDaniel Geißler
Daniel Geißler
BZBo Zhou
Bo Zhou
PLPaul Lukowicz
Paul Lukowicz
BGBerit Greinke
Berit Greinke
In this work, we propose a novel single-end morphing capacitive sensing method for shape tracking, FxC, by combining Folding origami structures and Capacitive sensing to detect the morphing structural motions using state-of-the-art sensing circuits and deep learning. It was observed through embedding areas of origami structures with conductive materials as single-end capacitive sensing patches, that the sensor signals change coherently with the motion of the structure. Different from other origami capacitors where the origami structures are used in adjusting the thickness of the dielectric layer of double-plate capacitors, FxC uses only a single conductive plate per channel, and the origami structure directly changes the geometry of the conductive plate. We examined the operation principle of morphing single-end capacitors through 3D geometry simulation combined with physics theoretical deduction, which deduced similar behavior as observed in experimentation. Then a software pipeline was developed to use the sensor signals to reconstruct the dynamic structural geometry through data-driven deep neural network regression of geometric primitives extracted from vision tracking. We created multiple folding patterns to validate our approach, based on folding patterns including Accordion, Chevron, Sunray and V-Fold patterns with different layouts of capacitive sensors using paper-based and textile-based materials. Experimentation results show that the geometry primitives predicted from the capacitive signals have a strong correlation with the visual ground truth with R-squared value of up to 95% and tracking error of 6.5 mm for patches. The simulation and machine learning constitute two-way information exchange between the sensing signals and structural geometry. By embedding part of the origami surface with morphing single-end capacitive sensors, FxC presents a unique solution that leverages both the mechanical properties of origami and sensing properties of capacitive sensing.

5 October 2024
Verified
Conference Paper
Open access
VRVitor Fortes Rey
Vitor Fortes Rey
LRLala Shakti Swarup Ray
Lala Shakti Swarup Ray
QXQingxin Xia
Qingxin Xia
KWKaishun Wu
Kaishun Wu
Paul LukowiczPaul Lukowicz
Paul Lukowicz
Due to the scarcity of labeled sensor data in HAR, prior research has turned to video data to synthesize Inertial Measurement Units (IMU) data, capitalizing on its rich activity annotations. However, generating IMU data from videos presents challenges for HAR in real-world settings, attributed to the poor quality of synthetic IMU data and its limited efficacy in subtle, fine-grained motions. In this paper, we propose Multi³Net, our novel multi-modal, multitask, and contrastive-based framework approach to address the issue of limited data. Our pretraining procedure uses videos from online repositories, aiming to learn joint representations of text, pose, and IMU simultaneously. By employing video data and contrastive learning, our method seeks to enhance wearable HAR performance, especially in recognizing subtle activities. Our experimental findings validate the effectiveness of our approach in improving HAR performance with IMU data. We demonstrate that models trained with synthetic IMU data generated from videos using our method surpass existing approaches in recognizing fine-grained activities.

2 December 2024
Verified
Book Chapter
Closed access
PDF file
LRLala Shakti Swarup Ray
Lala Shakti Swarup Ray
DGDaniel Geißler
Daniel Geißler
MLMengxi Liu
Mengxi Liu
BZBo Zhou
Bo Zhou
SSSungho Suh
Sungho Suh
PLPaul Lukowicz
Paul Lukowicz