To cultivate professional sports referees, we develop a sports referee training system, which can recognize whether a trainee wearing the Myo armband makes correct judging signals while watching a prerecorded professional game. The system has to correctly recognize a set of gestures related to official referee's signals (ORSs) and another set of gestures used to intuitively interact with the system. These two gesture sets involve both large motion and subtle motion gestures, and the existing sensor-based methods using handcrafted features do not work well on recognizing all kinds of these gestures. In this work, deep belief networks (DBNs) are utilized to learn more representative features for hand gesture recognition, and selective handcrafted features are combined with the DBN features to achieve more robust recognition results. Moreover, a hierarchical recognition scheme is designed to first recognize the input gesture as a large or subtle motion gesture, and the corresponding classifiers for large motion gestures and subtle motion gestures are further used to obtain the final recognition result. Moreover, the Myo armband consists of eight-channel surface electromyography (sEMG) sensors and an inertial measurement unit (IMU), and these heterogeneous signals can be fused to achieve better recognition accuracy. We take basketball as an example to validate the proposed training system, and the experimental results show that the proposed hierarchical scheme considering DBN features of multimodality data outperforms other methods.

Acupuncture therapy is one of the main modalities of treatment in Traditional Chinese Medicine (TCM). Based on different symptoms of the patient, needling or massaging is applied to the corresponding acupuncture points to relieve the symptoms. However, given the large number of acupuncture points and the complexity of their specifities, it is difficult for one to remember the location and function of each acupuncture point without extensive training. In this work, through the use of augmented reality (AR), the acupuncture points can be displayed directly on the image of human body. Compared to existing acupoint probe devices that work by measuring the skin conductivity, our solution does not require any additional hardware and is purely software-based. In this paper, we propose a novel approach for acupoint localization by leveraging the landmark points and 3D morphable model (3DMM). The localization error of our system is about 2.4mm which outperforms the existing work on acupoint localization by 170% in terms of estimation precision. A prototype system is implemented on the Android phone. In the case of mild symptoms (e.g. headache, sleep disorder), with the aid of our proposed system, the patient can quickly locate the corresponding acupuncture points for the application of massage to relieve his/her symptoms without the help from TCM physicians.

Vision Training is important for basketball players to effectively search for teammates who has wide-open opportunities to shoot, observe the defenders around the wide-open teammates and quickly choose a proper way to pass the ball to the most suitable one. We develop an immersive virtual reality (VR) system called VisionCoach to simulate the player's viewing perspective and generate three designed systematic vision training tasks to benefit the cultivating procedure. By recording the player's eye gazing and dribbling video sequence, the proposed system can analyze the vision-related behavior to understand the training effectiveness. To demonstrate the proposed VR training system can facilitate the cultivation of vision ability, we recruited 14 experienced players to participate in a 6-week between-subject study, and conducted a study by comparing the most frequently used 2D vision training method called Vision Performance Enhancement (VPE) program with the proposed system. Qualitative experiences and quantitative training results are reported to show that the proposed immersive VR training system can effectively improve player's vision ability in terms of gaze behavior and dribbling stability. Furthermore, training in the VR-VisionCoach Condition can transfer the learned abilities to real scenario more easily than training in the 2D-VPE Condition.