Shadow puppetry, a culturally rich storytelling art, faces challenges transitioning to the digital realm. Creators in the early design phase struggle with crafting intricate patterns, textures, and basic animations while adhering to stylistic conventions - hindering creativity, especially for novices. This paper presents Lumina, a tool to facilitate the early Chinese shadow puppet design stage. Lumina provides contour templates, animations, scene editing tools, and machine-generated traditional puppet patterns. These features liberate creators from tedious tasks, allowing focus on the creative process. Developed based on a formative study with puppet creators, the web-based Lumina enables wide dissemination. An evaluation with 18 participants demonstrated Lumina’s effectiveness and ease of use, with participants successfully creating designs spanning traditional themes to contemporary and science-fiction concepts.
Shadow puppetry is a cultural heritage with thousands of years of history, and its digital form has become a popular style in modern artistic creation. Creating high-quality shadow puppet works is often challenging. We present "ShadowMaker," an intelligent collaborative tool, where designers can co-create dynamic digital shadow puppets together with AI through prompts and sketching. ShadowMaker introduces a novel non-linear workflow, allowing users to freely switch between stages like sketching, layout, and animating.The user study with 18 participants demonstrated that ShadowMaker is easy to use, offers a high degree of creative freedom, and is capable of inspiring users’ creativity and inspiration.
Digital Musical Instruments (DMIs) have been integrated into group music therapy, providing therapists with alternative ways to engage in musical dialogues with their clients. However, existing DMIs used in group settings are primarily designed for individual use and often overlook the social dynamics inherent in group therapy. Recognizing the crucial role of social interaction in the effectiveness of group therapy, we argue that Collaborative Digital Musical Instruments (CDMIs), seamlessly integrating social interaction with musical expression, hold significant potential to enhance group music therapy. To better tailor CDMIs for group music therapy, we engaged in a co-design process with music therapists, designing and practicing group therapy sessions involving the prototype ComString. In the end, we reflected on the co-design case to suggest future directions for designing CDMIs in group music therapy.
Intangible Cultural Heritages (ICHs) of China span a wide range from handcrafts to folk performing arts. Although a variety of digital technologies are brought to support the inheritance and dissemination of ICHs in medias and museums, many technologies focus on video recording and displaying and lack of emerging experiences. As one of the famous Chinese ICHs, Xiàngshēng aka Crosstalk, is a folk stand-up comedy art with unique performing styles. It emphasizes facial and voice expressions. In this study, we propose a design method of robotic Xiàngshēng theater, in which two audio-animatronics actors – a wisecracker(dòu gén) and a straight man(pěng gén) – perform traditional Xiàngshēng art automatically. We explored cartoon-style animatronics character design with considerations of cultural and scenic factors, and implemented two audio-animatronics robots with a height of 1.5 meters and DOFs of 15 for each. Moving parts, like eyebrows and cheeks, are attached to enhance facial expressions of robot actors. Pre-recorded voice audio by professional human actors are highly expressive, requiring corresponding facial expressions with high expressiveness and accurate timing and tempo. In addition, traditional performing skills of Xiàngshēng require unique treats on presentations, such as exaggeration of facial expression, variation of tempo or logic semantics, etc. We propose a number of design schemes for robot actors, in accordance to different Xiàngshēng performing skills. We conducted a pilot study of expert interviews to evaluate the performance of proposed design schemes. Based on these design considerations, we built an actual robotic Xiàngshēng theater in an ICH museum of China. Robot actors will perform a 20 mins long Xiàngshēng comedy twice everyday to the public visitors. We conducted observation studies and interviews of the audience. Findings show a positive result for the robotic Xiàngshēng theater.
Inappropriate child-feeding behaviors are one of the causes of childhood obesity or stomach problems. Children from 1 to 5 years old don't know how much they should eat and parents may not know whether their children are full or not. Therefore, we designed a robot in a parent-child feeding scenario that visualizes children's daily diet through changes in its form. It aims to establish a more intuitive connection between scientific quantitative standards and actual feeding scenarios in the form of peripheral interaction. Thus solving the problem of real-time feedback in feeding.