Gesture fingertip detection plays a vital role in human-computer interaction applications such as VR and robot control. To improve the accuracy of fingertip detection in complex background, this paper proposes a fingertip detection algorithm based on a novel maximum discrimination HOG feature. Firstly, the Holistically-nested edge detection algorithm is used to detect the edge of hand images and non-hand images, and the HOG features of the contour edge are extracted to obtain the training set of positive and negative samples to reduce the influence of illumination, color, and texture on fingertip detection. Secondly, the maximum discrimination features are filtered from the positive sample set by a custom maximum discrimination feature filter and stored as a feature dictionary. The filtered maximum discrimination features and negative sample set are input into the XGBoost classifier for training and the voting rights classifier is obtained. Filtering maximum discrimination features can reduce the interference of irrelevant features and improve the performance of the classifier. Then, the KNN algorithm is used to find the best match in the dictionary, and the final fingertip position is obtained by the Meanshift algorithm. Finally, the algorithm in this paper is tested. The test results show that the accuracy of fingertip detection in this paper can reach 99 %, and the detected RMSE is kept in the range of 5 pixels, which is higher than that based on YCbCr skin color segmentation, YOLO target detection, and YOLO-YCbCr.

This work provides a 3D hand attitude estimation approach for fixed hand posture based on a CNN and LightGBM for dual-view RGB images to facilitate the application of hand posture teleoperation. First, using dual-view cameras and an IMU sensor, we provide a simple method for building 3D hand posture datasets. This method can quickly acquire dual-view 2D hand image sets and automatically append the appropriate three-axis attitude angle labels. Then, combining ensemble learning, which has strong regression fitting capabilities, with deep learning, which has excellent automatic feature extraction capabilities, we present an integrated hand attitude CNN regression model. This model uses a Bayesian optimization based LightGBM in the ensemble learning algorithm to produce 3D hand attitude regression and two CNNs to extract dual-view hand image features. Finally, a mapping from dual-view 2D images to 3D hand attitude angles is established using a training approach for feature integration, and a comparative experiment is run on the test set. The results of the experiments demonstrate that the suggested method may successfully solve the hand self-occlusion issue and accomplish 3D hand attitude estimation using only two normal RGB cameras.

In order to improve the real-time and accuracy of gesture position information acquisition in gesture interaction, this paper proposes a gesture position detection method based on generative adversarial networks. Firstly, this paper collects gesture images from two perspectives by binocular camera, and then uses semi-supervised generative adversarial network to segment gesture images. This method overcomes the problem that traditional semantic segmentation ignores the correlation between pixels, and obtains better gesture segmentation images. Secondly, fingertip detection and three-dimensional reconstruction of fingertips are performed on gesture semantic segmentation images. Finally, the method is verified by experiments, and the gesture segmentation results and fingertip detection results are evaluated by using mIoU value and fingertip detection accuracy. The experimental results show that the mIoU value of gesture segmentation can reach 94.8, and the accuracy of fingertip detection can reach 96.5.