Studies of Generative AI (GenAI)-assisted creative workflows have focused on individuals overcoming challenges of prompting to produce what they envisioned. When designers work in teams, how do collaboration and prompting influence each other, and how do users perceive generative AI and their collaborators during the co-prompting process? We engaged students with design or performance backgrounds, and little exposure to GenAI, to work in pairs with GenAI to create stage designs based on a creative theme. We found two patterns of collaborative prompting focused on generating story descriptions first, or visual imagery first. GenAI tools helped participants build consensus in the task, and allowed for discussion of the prompting strategies. Participants perceived GenAI as efficient tools rather than true collaborators, suggesting that human partners reduced the reliance on their use. This work highlights the importance of human-human collaboration when working with GenAI tools, suggesting systems that take advantage of shared human expertise in the prompting process.
2 May 2025 in Proceedings of the ACM on Human-Computer Interaction
Visitors to cultural heritage sites often encounter official information, while local people's unofficial stories remain invisible. To explore expression of local narratives, we conducted a workshop with 20 participants utilizing Generative AI (GenAI) to support visual narratives, asking them to use Stable Diffusion to create images of familiar cultural heritage sites, as well as images of unfamiliar ones for comparison. The results revealed three narrative strategies and highlighted GenAI's strengths in illuminating, amplifying, and reinterpreting personal narratives. However, GenAI showed limitations in meeting detailed requirements, portraying cultural features, and avoiding bias, which were particularly pronounced with unfamiliar sites due to participants' lack of local knowledge. To address these challenges, we recommend providing detailed explanations, prompt engineering, and fine-tuning AI models to reduce uncertainties, using objective references to mitigate inaccuracies from participants' inability to recognize errors or misconceptions, and curating datasets to train AI models capable of accurately portraying cultural features.
14 October 2024 in Proceedings of the ACM on Human-Computer Interaction
Character design in games involves interdisciplinary collaborations, typically between designers who create the narrative content, and illustrators who realize the design vision. However, traditional workflows face challenges in communication due to the differing backgrounds of illustrators and designers, the latter with limited artistic abilities. To overcome these challenges, we created Sketchar, a Generative AI (GenAI) tool that allows designers to prototype game characters and generate images based on conceptual input, providing visual outcomes that can give immediate feedback and enhance communication with illustrators' next step in the design cycle. We conducted a mixed-method study to evaluate the interaction between game designers and Sketchar. We showed that the reference images generated in co-creating with Sketchar fostered refinement of design details and can be incorporated into real-world workflows. Moreover, designers without artistic backgrounds found the Sketchar workflow to be more expressive and worthwhile. This research demonstrates the potential of GenAI in enhancing interdisciplinary collaboration in the game industry, enabling designers to interact beyond their own limited expertise.
Cultural Heritage is not just about tangible artifacts; it also includes intangible elements such as personal memories, community ties, and envisioned futures. Traditional museums and archives often emphasize physical items like architectural pieces and photos, while overlooking people’s personal and emotional connections to cultural heritage. To illustrate the personal connections people have with cultural heritage sites, we designed an exhibition that displayed images created by participants, which represent their perspectives and future visions of cultural heritage sites. The exhibition’s images, generated through GenAI, helped participants narratively describe cultural heritage locations, allowing them to express their visions of future threats like over-tourism and climate change on these sites. Contrary to constraints, co-creating with Generative AI associates participants with personal memories of cultural heritage, stimulating personal narratives and promoting deep reflection on cultural heritage preservation. The dissemination strategies we designed illustrate the use of GenAI to empower the expression of matters of cultural value beyond the physical.
Increasingly popular social virtual reality (VR) platforms like VRChat created new ways for people to interact with each other, generating dedicated user communities with unique idioms of socializing in an alternative world. In VRChat, users frequently gather in front of mirrors en masse during online interactions. Understanding how user communities deal with the mirror’s unique interactions can generate insights for supporting communication in social VR. In this study, we investigated the mirror’s synergistic effect with avatars on behaviors and dedicated user conversational performance. Qualitative findings indicate that avatar-mediated communication through mirrors provides functions like ensuring synchronization of incarnations, increasing immersion, and enhancing idealized embodiment to express bolder behaviors anonymously. Quantitative studies show that while mirrors improve self-perception, it has a potentially adverse effect on conversational performance, similar to the role of self-viewing in video conferencing. Studying how users interact with mirrors in an immersive environment allows us to explore how digital environments affect spatialized interactions when transported from physical to digital domains.
Policy mandates in addressing climate change are hindered by a lack of intrinsic motivation amongst participants to take collective action. Instead of overt persuasion, this study applied generative AI tools to speculative imagining of future climate scenarios and their adaptation strategies, using a workshop to encourage participants to align themselves with climate action. Participants used text-to-image tools to generate visions of the future in speculative scenarios, then prompted ChatGPT for potential solutions in these scenarios. They then asked text-to-image again to visualize the ChatGPT suggestions. Participants encountered difficulties editing or removing visual elements, dealt with the lack of transparency in the generation process by specifying the physical layout as opposed to the semantics, and collaboratively developed linguistic strategies for visual depiction of novel artifacts. This work shows how generative tools can be used to prototype future scenarios and envision designs that serve social purposes.