This paper presents the multi-task Deconvolutional Single Shot Detector (MT-DSSD), which runs three tasks—object detection, semantic object segmentation, and grasping detection for a suction cup—in a single network based on the DSSD. Simultaneous execution of object detection and segmentation by multi-task learning improves the accuracy of these two tasks. Additionally, the model detects grasping points and performs the three recognition tasks necessary for robot manipulation. The proposed model can perform fast inference, which reduces the time required for grasping operation. Evaluations using the Amazon Robotics Challenge (ARC) dataset showed that our model has better object detection and segmentation performance than comparable methods, and robotic experiments for grasping show that our model can detect the appropriate grasping point.
Student performance prediction was deployed to predict learning performance to identify at-risk students and provide interventions for them. However, prediction models should also consider external factors along with learning activities, such as course duration. Thus, we aim to distinguish the difference factor between the time dimension (duration of the course) and the feature dimension (students’ learning activities) by attention weights to provide helpful information and improve predictions of student performance. In this study, we introduce Attention-Based Artificial Neural Network (Attn-ANN), a novel model in educational data mining. The Attn-ANN combines attention weighting on the time and feature dimensions to examine the significance of lectures and learning activities and makes predictions by visualizing attention weight. We found that the Attn-ANN had a better area under the curve scores than conventional algorithms, and the attention mechanism allowed models to focus on input selectively. Incorporating the attention weighting of both the time and feature dimensions improved the prediction performance in an ablation study. Finally, we investigated and analyzed the model’s decision, finding that the Attn-ANN may be able to create synergy in real-world scenarios between the Attn-ANN’s predictions and instructors’ expertise, which underscores a novel contribution to engineering applications for interventions for at-risk students.
Hand posture recognition has tremendous potential in the field of natural user interactions. There were many advances in research in recent years but there are still limitations regarding its usage in unfavorable live situations where hand posture variation, illumination change or background complexity are an issue. In cases like these, recognizing the hand posture is a difficult task. As such, we considered reducing the difficulty of the task by using curriculum learning with intermediate information. We proceeded to divide the complex architecture of the hand posture recognition task into two easier ones: 1) Extraction of the hand shape under clutter background with illumination change, 2) Recognition of the hand posture from a binary image. In order to do so, we propose here a bottom-up structured deep convolutional neural network incorporating a special layer for binary image extraction. Our proposed method also employs state-of-the art techniques for deep learning to obtain generalization. As a result, we achieved better recognition performances of the hand posture under clutter background compared to the baseline method.
Action quality assessment (AQA) tasks in computer vision evaluate action quality in videos, and they can be applied to sports for performance evaluation. A typical example of AQA is predicting the final score from a video that captures an entire figure skating program. However, no previous studies have predicted individual jump scores, which are of great interest to competitors because of the high weight of competition. Despite the presence of unnecessary information in figure skating videos, human specialists can focus and reduce information when they evaluate jumps. In this study, we clarified the eye movements of figure skating judges and skaters while evaluating jumps and proposed a prediction model for jump performance that utilized specialists’ gaze location to reduce information. Kinematic features obtained from the tracking system were input into the model in addition to videos to improve accuracy. The results showed that skaters focused more on the face, whereas judges focused on the lower extremities. These gaze locations were applied to the model, which demonstrated the highest accuracy when utilizing both specialists’ gaze locations. The model outperformed human predictions and the baseline model (RMSE:0.775), suggesting a combination of human specialist knowledge and machine capabilities could yield higher accuracy.