20 November 2020 in IEEE Access
There are two basic problems in sign language recognition (SLR): (a) isolated word SLR and (b) continuous SLR. Most of the existing continuous SLR methods are extensions of the isolated word SLR methods. These methods use the isolated word SLR results as the basic module and obtain the sentence recognition results through sentence segmentation and word alignment. However, sentence segmentation and word alignment are often not accurate, resulting in a low sentence recognition accuracy. At the same time, continuous SLR usually requires strict sample labels, leading to the difficult task of manual labeling and limited training data availability. To address these challenges, this paper proposes a bidirectional spatial–temporal LSTM fusion attention network (Bi-ST-LSTM-A) for continuous SLR. This approach avoids problems such as sentence segmentation, word alignment, and tedious manual labeling. Our contributions are summarized as follows: (1) we proposed a sign language video feature representation method using a convolutional neural network (CNN) and spatial–temporal LSTM (ST-LSTM) information fusion technology; and (2) we constructed a uniform neural machine translation framework that can be used for complex continuous SLR and gesture recognition of nonspecific people in nonspecific environments. Experiments were carried out on some large continuous sign language datasets. The sign language recognition accuracy reached 81.22% on the 500 CSL dataset, 76.12% on the RWTH-PHOENIX-Weather dataset and 75.32% on the RWTH-PHOENIX-Weather-2014T dataset, thereby illustrating the effectiveness of the proposed framework.
7 September 2021 in IEEE Access
At present, the theory and application of fractional-order neural networks remain in the exploratory stage. We study the asymptotic stability of fractional-order neural networks with Riemann-Liouville (R-L) derivatives. For non-delayed and delayed systems, we propose an asymptotic stability criterion based on the combination of the Lyapunov method and linear matrix inequality (LMI) method. The highlights include the following: (1) for fractional-order neural networks with time delay, the existence and uniqueness of solutions are proven by using matrix analysis theory and contraction mapping theorem, and (2) based on the unique solution, a suitable Lyapunov functional is constructed. Based on the inequality theorem and LMI method, two sets of asymptotic stability criteria for fractional-order neural networks are proven, which avoids the difficulty of solving the fractional derivative by the Leibniz law. Finally, the results are verified using numerical simulations.