In order to recreate viable and human-like conversational responses, the artificial entity, i.e., an embodied conversational agent, must express correlated speech (verbal) and gestures (non-verbal) responses in spoken social interaction. Most of the existing frameworks focus on intent planning and behavior planning. The realization, however, is left to a limited set of static 3D representations of conversational expressions. In addition to functional and semantic synchrony between verbal and non-verbal signals, the final believability of the displayed expression is sculpted by the physical realization of non-verbal expressions. A major challenge of most conversational systems capable of reproducing gestures is the diversity in expressiveness. In this paper, we propose a method for capturing gestures automatically from videos and transforming them into 3D representations stored as part of the conversational agent’s repository of motor skills. The main advantage of the proposed method is ensuring the naturalness of the embodied conversational agent’s gestures, which results in a higher quality of human-computer interaction. The method is based on a Kanade–Lucas–Tomasi tracker, a Savitzky–Golay filter, a Denavit–Hartenberg-based kinematic model and the EVA framework. Furthermore, we designed an objective method based on cosine similarity instead of a subjective evaluation of synthesized movement. The proposed method resulted in a 96% similarity.

This paper presents a method of binocular visual stimulation for brain–computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEPs) using phase-coded symbols. The proposed method’s emphasis is on a binocular phase-coded visual stimulus, which is based on the phase differences between the left- and right-eye stimuli, and a symbol detection and recognition procedure based on SSVEP response of the left and right occipital lobes of the user’s scalp, where the SSVEP response is obtained as electroencephalography (EEG) signaling. The symbols are coded as phase differences and maintain the same frequency of the sine wave-modulated light provided to the user’s left and right eyes as a binocular visual stimulation. Based on this method, a basic system setup is presented to explore the possibilities of binocular phase-coded visual stimuli for virtual or augmented reality applications, where the binocular visual stimulation was achieved by the specially designed head-mounted displays. Multiple visually coded targets are realized as eight different phase-coded binocular symbols and further evaluated as a random sequence of single targets, thus representing the situations in virtual or augmented reality, where multiple visually coded targets are present but not visualized to the user simultaneously within the same field of view. The offline results obtained from ten healthy subjects revealed that an average symbol recognition accuracy of 90.63% and an information transfer rate (ITR) of 70.55 bits/min were achieved for a symbol stimulation time of 2 s. The results of this paper demonstrate the feasibility of using binocular visual stimuli for SSVEP-based BCIs, where reasonable ITR is achieved using single-frequency binocular phase-coded symbols. The proposed method indicates the possibility of combining it with 3D wearable visualization technologies, such as binocular head-mounted displays (HMDs), in order to improve the intuitiveness of the interaction with more immersive user experience using BCI modalities.

The incorporation of grammatical information into speech recognition systems is often used to increase performance in morphologically rich languages. However, this introduces demands for sufficiently large training corpora and proper methods of using the additional information. In this paper, we present a method for building factored language models that use data obtained by morphosyntactic tagging. The models use only relevant factors that help to increase performance and ignore data from other factors, thus also reducing the need for large morphosyntactically tagged training corpora. Which data is relevant is determined at run-time, based on the current text segment being estimated, i.e., the context. We show that using a context-dependent model in a two-pass recognition algorithm, the overall speech recognition accuracy in a Broadcast News application improved by 1.73% relatively, while simpler models using the same data achieved only 0.07% improvement. We also present a more detailed error analysis based on lexical features, comparing first-pass and second-pass results.