Given the achievements in automatically translating text from one language to another, one would expect to see similar advancements in translating between signed and spoken languages. However, progress in this effort has lagged in comparison. Typically, machine translation consists of processing text from one language to produce text in another. Because signed languages have no generally-accepted written form, translating spoken to signed language requires the additional step of displaying the language visually as animation through the use of a three-dimensional (3D) virtual human commonly known as an avatar. Researchers have been grappling with this problem for over twenty years, and it is still an open question. With the goal of developing a deeper understanding of the challenges posed by this question, this article gives a summary overview of the unique aspects of signed languages, briefly surveys the technology underlying avatars and performs an in-depth analysis of the features in a textual representation for avatar display. It concludes with a comparison of these features and makes observations about future research directions.
Sign Languages (SLs) are visual–gestural languages that have developed naturally in deaf communities. They are based on the use of lexical signs, that is, conventionalized units, as well as highly iconic structures, i.e., when the form of an utterance and the meaning it carries are not independent. Although most research in automatic Sign Language Recognition (SLR) has focused on lexical signs, we wish to broaden this perspective and consider the recognition of non-conventionalized iconic and syntactic elements. We propose the use of corpora made by linguists like the finely and consistently annotated dialogue corpus Dicta-Sign-LSF-v2. We then redefined the problem of automatic SLR as the recognition of linguistic descriptors, with carefully thought out performance metrics. Moreover, we developed a compact and generalizable representation of signers in videos by parallel processing of the hands, face and upper body, then an adapted learning architecture based on a Recurrent Convolutional Neural Network (RCNN). Through a study focused on the recognition of four linguistic descriptors, we show the soundness of the proposed approach and pave the way for a wider understanding of Continuous Sign Language Recognition (CSLR).
1 January 2018 in TIPA. Travaux interdisciplinaires sur la parole et le langage
Cet article relate une étude qui porte sur l'analyse linguistique de certains proformes à partir de l’annotation d’un corpus de Langue des Signes Française (LSF) nommé MOCAP1. Nous nous sommes plus particulièrement intéressés aux proformes qui servent à introduire un référent du discours, et nous avons analysé trois structures mettant en œuvre respectivement le maintien d’une configuration avec la main dominée, la séquentialité et l’utilisation de spécifieurs non manuels tels que la labialisation. Nous avons tenté de décrire les formes récurrentes observées pour ces trois structures avec le formalisme AZEE qui prend en compte les particularités des langues signées. Ce même formalisme a également pour vocation d’être utilisé pour le traitement automatique des langues des signes.