Brian P. Bailey

Brian P. Bailey

Auto-generated profile, based on publicly available information

Brian P. Bailey

University of Illinois, Urbana-Champaign, Urbana, IL, USA

Research Focus & Metrics

Academic Career Length

24Years (2003-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 April 2023 in Proceedings of the ACM on Human-Computer Interaction
Verified
Journal Article
Open access
HDHyo Jin Do
Hyo Jin Do
HKHa-Kyung Kong
Ha-Kyung Kong
PTPooja Tetali
Pooja Tetali
JLJaewook Lee
Jaewook Lee
BBBrian P. Bailey
Brian P. Bailey
Conversational agents (CAs) can analyze online conversations using natural language techniques and effectively facilitate group discussions by sending supervisory messages. However, if a CA makes imperfect interventions, users may stop trusting the CA and discontinue using it. In this study, we demonstrate how inaccurate interventions of a CA and a conversational repair strategy can influence user acceptance of the CA, members' participation in the discussion, perceived discussion experience between the members, and group performance. We built a CA that encourages the participation of members with low contributions in an online chat discussion in which a small group (3-6 members) performs a decision-making task. Two types of errors can occur when detecting under-contributing members: 1) false-positive (FP) errors happen when the CA falsely identifies a member as under-contributing and 2) false-negative (FN) errors occur when the CA misses detecting an under-contributing member. We designed a conversational repair strategy that gives users a chance to contest the detection results and the agent sends a correctional message if an error is detected. Through an online study with 175 participants, we found that participants who received FN error messages reported higher acceptance of the CA and better discussion experience, but participated less compared to those who received FP error messages. The conversational repair strategy moderated the effect of errors such as improving the perceived discussion experience of participants who received FP error messages. Based on our findings, we offer design implications for which model should be selected by practitioners between high precision (i.e., fewer FP errors) and high recall (i.e., fewer FN errors) models depending on the desired effects. When frequent FP errors are expected, we suggest using the conversational repair strategy to improve the perceived discussion experience.

4 June 2016
Verified
Conference Paper
Open access
YYYu-Chun (Grace) Yen
Yu-Chun (Grace) Yen
SDSteven P. Dow
Steven P. Dow
EGElizabeth Gerber
Elizabeth Gerber
BBBrian P. Bailey
Brian P. Bailey
Increasingly, designers seek feedback on their designs from crowd platforms such as social networks, Web forums, and paid task markets which demand different amounts of social capital, financial resources, and time. Yet it is unknown how the choice of crowd platform affects feedback generation. We conducted an online study where designers created initial designs and revised the designs based on crowd feedback. We measured the quantity, quality, and content of the feedback received at two iterations and from crowds driven by social status, enjoyment, and financial gain. Our results show, for example, that task markets yield more suggestions, online forums provide more process feedback, and social networks give the most suggestions without payment. We contribute an emergent framework for crowd feedback selection, opportunities for enhancing feedback services, and an experimental platform that researchers can adapt to reduce the burden of conducting online studies of design feedback.

17 June 2014 in Human–Computer Interaction
Verified
Journal Article
Closed access
SMScarlett R. Miller
Scarlett R. Miller
BBBrian P. Bailey
Brian P. Bailey
AKAlex Kirlik
Alex Kirlik
Expanding and improving design knowledge is a vital part of higher education due to the growing demand for employees who can think both critically and creatively. However, developing effective methods for assessing what students have learned in design courses is one of the most elusive challenges of design education due to the subjective nature of design. For example, evaluating design outcomes is problematic due to the common pattern of increasing enrollments and reduced resources for design instruction. In this article, we propose and evaluate a new assessment method that uses a novel application of Bayesian Truth Serum (BTS), a scoring algorithm, in order to provide a scalable and reliable measure of design knowledge. This method requires no subjective input from the design instructor, nor does it require answers to questions that have distinct right or wrong answers. We tested this method over a 4-week period with 71 design students in an upper-level design course. For the study, participants were asked to provide responses to multiple-choice BTS survey questions, generate ideas for a design problem, and provide feedback on other participants' ideas. The survey data were used to calculate BTS indices of expertise and statistical tests were performed to determine how the indices correlated with participant ideation and critique proficiency. The results from this study show a modest correlation between the BTS indices of expertise and later performance on generative design tasks and a correlation between the students' ability to critique designs and their BTS scores. These findings suggest that the BTS assessment method can be used to supplement existing evaluation practices for individual design assessment, particularly in courses where group projects are used as the primary means of evaluation. In addition, the results show promise for using the BTS method in classes where design projects or design critiques are not feasible due to time constraints or large class sizes.

28 September 2023 in Proceedings of the ACM on Human-Computer Interaction
Verified
Journal Article
Open access
HDHyo Jin Do
Hyo Jin Do
HKHa-Kyung Kong
Ha-Kyung Kong
PTPooja Tetali
Pooja Tetali
KKKarrie Karahalios
Karrie Karahalios
BBBrian P. Bailey
Brian P. Bailey
A conversational agent (CA) effectively facilitates online group discussions at scale. However, users may have expectations about how well the CA would perform that do not match with the actual performance, compromising technology acceptance. We built a facilitator CA that detects a member who has low contribution during a synchronous group chat discussion and asks the person to participate more. We designed three techniques to set end-user expectations about how accurately the CA identifies an under-contributing member: 1)information: explicitly communicating the accuracy of the detection algorithm, 2)explanation: providing an overview of the algorithm and the data used for the detection, and 3)adjustment: enabling users to gain a feeling of control over the algorithm. We conducted an online experiment with 163 crowdworkers in which each group completed a collaborative decision-making task and experienced one of the techniques. Through surveys and interviews, we found that the explanation technique was the most effective strategy overall as it reduced user embarrassment, increased the perceived intelligence of the CA, and helped users better understand the detection algorithm. In contrast, the information technique reduced members' contributions and the adjustment technique led to a more negative perceived discussion experience. We also discovered that the interactions with other team members diluted the effects of the techniques on users' performance expectations and acceptance of the CA. We discuss implications for better designing expectation-setting techniques for AI-team collaboration such as ways to improve collaborative decision outcomes and quality of contributions.

1 January 2006
Verified
Conference Paper
Open access
PDF file
JBJacob T. Biehl
Jacob T. Biehl
BBBrian P. Bailey
Brian P. Bailey
Productive collaboration in a multiple-device environment (MDE) requires an effective interface for efficiently managing applications among devices. Though many interfaces exist, there is little empirical understanding of how they affect collaboration. This paper reports results from a user study comparing how well three classes of interfaces; textual, map, and iconic, support application management during realistic, collaborative activities in an MDE. From empirical results, observations, and an analysis of how users interacted with each interface, we produced a set of design lessons for improving management interfaces. The lessons were demonstrated within the iconic interface, but they are just as applicable to other interfaces. This work contributes further understanding of how to design effective management interfaces for MDEs.

14 April 2023 in Proceedings of the ACM on Human-Computer Interaction
Verified
Journal Article
Open access
WSWenxuan Wendy Shi
Wenxuan Wendy Shi
SKSneha R. Krishna Kumaran
Sneha R. Krishna Kumaran
HSHari Sundaram
Hari Sundaram
BBBrian P. Bailey
Brian P. Bailey
Peer evaluations are a well-established tool for evaluating individual and team performance in collaborative contexts, but are susceptible to social and cognitive biases. Current peer evaluation tools have also yet to address the unique opportunities that online collaborative technologies provide for addressing these biases. In this work, we explore the potential of one such opportunity for peer evaluations: data traces automatically generated by collaborative tools, which we refer to as "activity traces". We conduct a between-subjects experiment with 101 students and MTurk workers, investigating the effects of reviewing activity traces on peer evaluations of team members in an online collaborative task. Our findings show that the usage of activity traces led participants to make more and greater revisions to their evaluations compared to a control condition. These revisions also increased the consistency and participants' perceived accuracy of the evaluations that they received. Our findings demonstrate the value of activity traces as an approach for performing more reliable and objective peer evaluations of teamwork. Based on our findings as well as qualitative analysis of free-form responses in our study, we also identify and discuss key considerations and design recommendations for incorporating activity traces into real-world peer evaluation systems.