Virtual human is widely employed in various industries, including personal assistance, intelligent customer service, and online education, thanks to the rapid development of artificial intelligence. An anthropomorphic digital human can quickly contact people and enhance user experience in human–computer interaction. Hence, we design the human–computer interaction system framework, which includes speech recognition, text-to-speech, dialogue systems, and virtual human generation. Next, we classify the model of talking-head video generation by the virtual human deep generation framework. Meanwhile, we systematically review the past five years’ worth of technological advancements and trends in talking-head video generation, highlight the critical works and summarize the dataset.

As a branch of sentiment analysis tasks, emotion recognition in conversation (ERC) aims to explore the hidden emotions of a speaker by analyzing the sentiments in utterance. In addition, emotion recognition in multimodal data from conversation includes the text of the utterance and its corresponding acoustic and visual data. By integrating features from various modalities, the emotion of utterance can be more accurately predicted. ERC research faces challenges in context construction, speaker dependency design, and multimodal heterogeneous feature fusion. Therefore, this review starts by defining the ERC task, developing the research work, and introducing the utilized datasets in detail. Simultaneously, we analyzed context modeling in conversations, speaker dependency, and methods for fusing multimodal information concerning existing research work for evaluation purposes. Finally, this review also explores the research, application challenges, and opportunities of ERC.

This paper proposes a neural generative architecture, namely NLDT, to generate a natural language short text describing a table which has formal structure and valuable information. Specifically, the architecture maps fields and values of a table to continuous vectors and then generates a natural language description by leveraging the semantics of a table. The NLDT architecture adopts a two-level neural model to make the most of the structure of a table to fully express the relationship between contents. To deal with the problem of out-of-vocabulary, this paper develops a simple and fast word-conversion method that replaces rare words appearing in texts with common field information in tables and directly replicates contents from table to the output sequence according to the field information. Besides, this paper adds the concept of theme to adapt the NLDT architecture to open domain and improves beam search algorithm to strengthen the results in the inference stage. On the WEATHERGOV dataset, the NLDT architecture improves the state-of-the-art BLEU-4 score from 61.01 to 62.89 and the current state-of-the-art F1 score from 73.21 to 78. On the WIKIBIO and WIKITABLE datasets, the NLDT architecture achieves a BLEU-4 score of 45.77 and 38.71 respectively which also outperform the state-of-the-art approaches. Furthermore, this paper introduces a Chinese dataset WIKIBIOCN including 33,244 biographies with corresponding tables. On the WIKIBIOCN dataset, the NLDT architecture achieves a BLEU-4 score of 38.87 and fairly good manual evaluation.

Virtual simulation can solve the challenges of high cost, long cycle time, and inaccessibility in traditional experimental teaching, which is far-reaching for talent training. This study combines bibliometric visualization theory with AHP (Analytic Hierarchy Process). It establishes a hierarchical evaluation model of a virtual simulation experimental teaching platform based on 842 questionnaires and 4787 articles, including 68,306 citation records, and deconstructing the complex evaluation problem into several multidimensional factors by attributes and relationships. Based on this, a virtual simulation experimental teaching platform construction scheme for IP protocol analysis based on a network covert communication perspective is outputted, which is compatible with the research results. The experimental platform takes a task-driven teaching method as the core, mainly including four modules of context creation, task determination, independent learning, and effect evaluation. The experience of building this platform can be extended to other disciplines, leading the teaching reform exploration of practice-based, innovation-focused, and engineering-critical, helping to implement the flipped classroom, and promoting the development of education modernization.

Audio-driven cross-modal talking head generation has experienced significant advancement in the last several years, and it aims to generate a talking head video that corresponds to a given audio sequence. Out of these approaches, the NeRF-based method can generate videos featuring a specific person with more natural motion compared to the one-shot methods. However, previous approaches failed to distinguish the importance of different regions, resulting in the loss of information-rich region features. To alleviate the problem and improve video quality, we propose MLDF-NeRF, an end-to-end method for talking head generation, which can achieve better vector representation through multi-level feature dynamic fusion. Specifically, we designed two modules in MLDF-NeRF to enhance the cross-modal mapping ability between audio and different facial regions. We initially developed a multi-level tri-plane hash representation that uses three sets of tri-plane hash networks with varying resolutions of limitation to capture the dynamic information of the face more accurately. Then, we introduce the idea of multi-head attention and design an efficient audio-visual fusion module that explicitly fuses audio features with image features from different planes, thereby improving the mapping between audio features and spatial information. Meanwhile, the design helps to minimize interference from facial areas unrelated to audio, thereby improving the overall quality of the representation. The quantitative and qualitative results indicate that our proposed method can effectively generate talk heads with natural actions and realistic details. Compared with previous methods, it performs better in terms of image quality, lip sync, and other aspects.

The current research on generative cross-lingual summarization models mainly adopts the framework of seq2seq, where entity relation errors are inevitable. How to reduce the occurrence of errors and improve the accuracy of summary generation needs to be addressed urgently. In this regard, we try to add the task of extracting entity relations as an auxiliary task in the Chinese-to-English cross-lingual text summarization model to reduce the entity relation errors in the summaries, and propose two fusion strategies for merging the auxiliary task. In addition, we also adjust the dataset for this task. The experiment proves that it is effective to reduce errors as well as improve the performance of cross-lingual text summarization model by merging the auxiliary task of extracting entity relations in the way of splicing summaries and entity relation triples as real labels of training sets.