As a promising component for body sensor networks, the wearable sensors for hand gesture recognition have increasingly received great attention in recent years. By interpreting human intentions through hand gestures, the natural human-robot interaction can be realized in the smart home where the youth and the elderly can perform hand gestures to control the household robot or the robotic wheelchair. Here, a wearable wrist-worn camera sensor (WristCam) was shown to recognize hand trajectory gestures. The moving velocity of the user's hand was deduced from the matched Speeded Up Robust Features (SURF) keypoints of the moving background of the video sequence. Furthermore, the segmentation of continuous gestures was achieved by detecting the predefined gesture starting signal from the hand region of the image, which was segmented by the Lazy Snapping algorithm. In our study, 10 types of gestures and 1350 gesture samples collected from 15 subjects at 3 different scenes were classified by the dynamic time warping (DTW) algorithm and the results achieved an average recognition accuracy up to 97.6%. Moreover, the practicability of the proposed system was further demonstrated by controlling a cooperative robot to draw letters on paper.
The arising of domestic robots in smart infrastructure has raised demands for intuitive and natural interaction between humans and robots. To address this problem, a wearable wrist-worn camera (WwwCam) is proposed in this paper. With the capability of recognizing human hand gestures in real-time, it enables services such as controlling mopping robots, mobile manipulators, or appliances in smart-home scenarios. The recognition is based on finger segmentation and template matching. Distance transformation algorithm is adopted and adapted to robustly segment fingers from the hand. Based on fingers’ angles relative to the wrist, a finger angle prediction algorithm and a template matching metric are proposed. All possible gesture types of the captured image are first predicted, and then evaluated and compared to the template image to achieve the classification. Unlike other template matching methods relying highly on large training set, this scheme possesses high flexibility since it requires only one image as the template, and can classify gestures formed by different combinations of fingers. In the experiment, it successfully recognized ten finger gestures from number zero to nine defined by American Sign Language with an accuracy up to 99.38%. Its performance was further demonstrated by manipulating a robot arm using the implemented algorithms and WwwCam to transport and pile up wooden building blocks.