To promote data transparency, frameworks such as CrowdWorkSheets encourage documentation of annotation practices on the interfaces of AI systems, but we do not know how they affect user experience. Will the quality of labeling affect perceived credibility of training data? Does the source of annotation matter? Will a credible dataset persuade users to trust a system even if it shows racial biases in its predictions? To find out, we conducted a user study (N = 430) with a prototype of a classification system, using a 2 (labeling quality: high vs. low) × 4 (source: others-as-source vs. self-as-source cue vs. self-as-source voluntary action, vs. self-as-source forced action) × 3 (AI performance: none vs. biased vs. unbiased) experiment. We found that high-quality labeling leads to higher perceived training data credibility, which in turn enhances users’ trust in AI, but not when the system shows bias. Practical implications for explainable and ethical AI interfaces are discussed.
Prompts are the primary medium for interacting with generative AI tools. However, users often lack sufficient prompt literacy and motivation to fully benefit from these tools. To address this, we explore whether introducing prompt coaching into a chatbot-based generative AI interface can influence users’ perceptions and engagement of prompting, and further affect their trust in the system. In a user study (N = 132), we found that prompt coaching encourages users to specify more details in their prompts, even though over half initially believed their prompts were sufficient. Furthermore, the coach increased users’ cognitive elaboration, which was associated with higher perceived trust calibration. However, prompt coaching did not significantly enhance UX, although users in the coaching absent condition expressed a strong need for prompt assistance for better user experience. These findings have practical implications for the design of trustworthy and responsible generative AI interfaces.