Gesture recognition (GR) has many applications for human-computer interaction (HCI) in the healthcare, home, and business arenas. However, the common techniques to realize gesture recognition using video processing are computationally intensive and expensive. In this work, we propose to task existing visible light communications (VLC) systems with gesture recognition. Different finger movements are identified by training on the light transitions between fingers using the long short-term memory (LSTM) neural network. This paper describes the design and implementation of the gesture recognition technique for a practical VLC system operating over a distance of 48 cm. The platform uses a single low-cost light-emitting diode (LED) and photo-diode sensor at the receiver side. The system recognizes gestures from interruptions in the direct light transmission, and is therefore suitable for high-speed communication. Gesture recognition accuracies were conducted for five gestures, and results demonstrate that the proposed system is able to accurately identify the gestures in up to 88% of cases.

Human activity recognition (HAR) employs machine learning for the automated recognition of motion and has widespread applications across healthcare, daily-life and security spaces. High performances have especially been demonstrated using video cameras and intensive signal processing such as the convolutional neural network (CNN). However, lower complexity algorithms operating on low-rate inertial data is a promising approach for portable use-cases such as pairing with smart wearables. This work considers the performance benefits from combining HAR classification estimates from multiple sensors each with lower-complexity processing compared with a higher-complexity single-sensor classifier. We show that while the highest single-sensor classification accuracy of 91% can be achieved for seven activities with optimized number of hidden units and sample rate, the classification accuracy is reduced to 56% with a reduced-complexity 50-neuron classifier. However, by majority combining the predictions of three and four low-complexity classifiers, the average classification accuracy increased to 82.5% and 94.4%, respectively, demonstrating the efficacy of this approach.

Hand gestures are a natural and efficient means to control systems and are one of the promising but challenging areas of human–machine interaction (HMI). We propose a system to recognize gestures by processing interrupted patterns of light in a visible light communications (VLC) system. Our solution is aimed at the emerging light communication systems and can facilitate the human–computer interaction for services in health-care, robot systems, commerce and the home. The system exploits existing light communications infrastructure using low-cost and readily available components. Different finger sequences are detected using a probabilistic neural network (PNN) trained on light transitions between fingers. A novel pre-processing of the sampled light on a photodiode is described to facilitate the use of the PNN with limited complexity. The contributions of this work include the development of a sensing technique for light communication systems, a novel PNN pre-processing methodology to convert the light sequences into manageable size matrices along with hardware implementation showing the proof of concept under natural lighting conditions. Despite the modest complexity our system could correctly recognize gestures with an accuracy of 73%, demonstrating the potential of this technology. We show that the accuracy depends on the PNN pre-processing matrix size and the Gaussian spread function. The emerging IEEE 802.11bb ‘Li-Fi’ standard is expected to bring the light communications infrastructure into virtually every room across the world and a methodology to exploit a system for gesture sensing is expected to be of considerable interest and value to society.