In this demo we showcase an interactive application to support the learning of “TikTok dance challenge” short dance choreographies. Our system utilizes dance challenge videos as the information source, performing music analysis and pose estimation to segment the dance into learnable chunks and generate a practice plan that implements motor learning techniques such as incremental part-learning and fading guidance. These plans are presented in a web app that implements video demonstration, augmented webcam mirroring, practice recording/review functionality, and both concurrent and terminal feedback. By operating on a ubiquitous information source, generating the lessons automatically, and requiring only a web browser and webcam in the user interface, our system is a step towards significantly expanding the reach of dance choreography learning and a platform for further research into dance HCI.
The presence of semantic information in multivariate patterns of neural activity has been explored as a method of measuring knowledge and learning. Using fMRI, we investigated whether novice learners of American Sign Language (ASL) showed overlapping representations of semantic categories for words presented in a well-known (English) or newly learned (ASL) language. We find evidence of neural patterns that were partially shared between sign and speech in novice participants. This result provides evidence for the influence of even brief learning on neural representations in cross-modality language processing.
Semantic concepts relate to each other to varying degrees to form a network of first-order relations, and these first-order relations serve as input into networks of general relation types as well as higher order relations. Previous work has studied the neural mapping of semantic concepts across domains, though much work remains to be done to understand how the localization and structure of those architectures differ depending on various individual differences in attentional bias towards different content presentation formats. Using an item-wise model of semantic distance of first-order relations (word2vec) between stimuli (presented both in word and picture forms), we used representational similarity analysis to identify individual differences in the neural localization of semantic concepts, and how those localization differences can be predicted by individual variance in the degree to which individuals attend to word information instead of pictures. Importantly, there were no reliable representations of this first-order semantic relational network when looking at the full group, and it was only through considering individual differences that a stable localization difference became evident. These results indicate that individual differences in the degree to which a person habitually attends to word information instead of picture information substantially affects the neural localization of first-order semantic representations.
How do novice language learners represent semantic information in their new language? The extent to which multiple languages are supported by divergent or overlapping semantic representations in bilinguals has been well-studied, but less is known about how new knowledge is integrated into established representational networks at the earliest stages of acquisition. Furthermore, examining language across modality (sign vs. speech) can provide unique insight into language unconfounded by perceptual features. We present two experiments in which hearing non-signers underwent brief training in American Sign Language (ASL) followed by fMRI scanning. Across both datasets (N=50), we use representational similarity analysis (RSA) to identify brain regions where neural patterns reflect semantic relationships between stimuli. In Study 2 (N=40) we show that multivariate neural measures of semantic representation in several frontal, temporal, and occipital regions reflect individual participant-level comprehension. These results demonstrate the role of frontal and temporal regions, especially bilateral superior temporal sulcus, in representing semantic content across language and modality in novice learners.