Haojian Jin

Haojian Jin

Auto-generated profile, based on publicly available information

Haojian Jin

University of California San Diego, La Jolla, California, USA

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
7 November 2024 in Proceedings of the ACM on Human-Computer Interaction
Verified
Journal Article
Open access
ZCZirui Cheng
Zirui Cheng
JXJingfei Xu
Jingfei Xu
HJHaojian Jin
Haojian Jin
The advances of generative AI have posed a challenge for using open-ended questions to assess conceptual learning outcomes, as it is increasingly common for students to use tools like ChatGPT to generate long textual answers. However, teachers still have to spend substantial time reading the answers and inferring students' learning outcomes. We present TreeQuestion, a human-in-the-loop system designed to help teachers create a set of multiple-choice questions to assess students' conceptual learning outcomes. When a teacher seeks to assess students' comprehension of specific concepts, TreeQuestion taps into the wealth of knowledge embedded within large language models and generates a set of multiple-choice questions organized in a tree-like structure. We evaluated TreeQuestion with 96 students and 10 teachers. Results indicated that students achieved similar performance in multiple-choice questions generated by TreeQuestion and open-ended questions graded by teachers. Meanwhile, TreeQuestion could reduce teachers' efforts in creating and grading the multiple-choice questions in contrast to manually generated open-ended questions. We estimate that in a hypothetical class with 20 students, using multiple-choice questions from TreeQuestion may require only 4.6% of the time compared to open-ended questions for assessing learning outcomes.

11 May 2024
Verified
Conference Paper
Open access
YYYaqing Yang
Yaqing Yang
TLTony W Li
Tony W Li
HJHaojian Jin
Haojian Jin
Predicting users’ privacy concerns is challenging due to privacy’s subjective and complex nature. Previous research demonstrated that generic attitudes, such as those captured by Westin’s Privacy Segmentation Index, are inadequate predictors of context-specific attitudes. We introduce ContextLabel, a method enabling practitioners to capture users’ privacy profiles across domains and predict their privacy concerns towards unseen data practices. ContextLabel’s key innovations are (1) using non-mutually exclusive labels to capture more nuances of data practices, and (2) capturing users’ privacy profiles by asking them to express privacy concerns to a few data practices. To explore the feasibility of ContextLabel, we asked 38 participants to express their thoughts in free text towards 13 distinct data practices across five days. Our mixed-methods analysis shows that a preliminary version of ContextLabel can predict users’ privacy concerns towards unseen data practices with an accuracy (73%) surpassing Privacy Segmentation Index (56%) and methods using categorical factors (59%).

21 November 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
QLQiyu Li
Qiyu Li
JWJinhe Wen
Jinhe Wen
HJHaojian Jin
Haojian Jin
A major concern with integrating large language model (LLM) services (e.g., ChatGPT) into workplaces is that employees may inadvertently leak sensitive information through their prompts. Since user prompts can involve arbitrary vocabularies, conventional data leak mitigation solutions, such as string-matching-based filtering, often fall short. We present GPTWall, a privacy firewall that helps internal admins create and manage policies to mitigate data leaks in prompts sent to external LLM services. GPTWall's key innovations are (1) introducing a lightweight LLM running on the edge to obfuscate target information in prompts and restore the information after receiving responses, and (2) helping admins author fine-grained disclosure policies through programming by example. We evaluated GPTWall with 12 participants and found that they could create an average of 17.7 policies within 30 minutes, achieving an increase of 29% in precision and 22% in recall over the state-of-the-art data de-identification tool.

2 December 2024
Verified
Conference Paper
Open access
PWPeiran Wang
Peiran Wang
QLQiyu Li
Qiyu Li
LYLongxuan Yu
Longxuan Yu
ZWZiyao Wang
Ziyao Wang
ALANG Li
ANG Li
HJHaojian Jin
Haojian Jin
We present Moderator, a policy-based model management system that allows administrators to specify fine-grained content moderation policies and modify the weights of a text-to-image (TTI) model to make it significantly more challenging for users to produce images that violate the policies. In contrast to existing general-purpose model editing techniques, which unlearn concepts without considering the associated contexts, Moderator allows admins to specify what content should be moderated, under which context, how it should be moderated, and why moderation is necessary. Given a set of policies, Moderator first prompts the original model to generate images that need to be moderated, then uses these self-generated images to reverse fine-tune the model to compute task vectors for moderation and finally negates the original model with the task vectors to decrease its performance in generating moderated content. We evaluated Moderator with 14 participants to play the role of admins and found they could quickly learn and author policies to pass unit tests in approximately 2.29 policy iterations. Our experiment with 32 stable diffusion users suggested that Moderator can prevent 65% of users from generating moderated content under 15 attempts and require the remaining users an average of 8.3 times more attempts to generate undesired content.

11 May 2024
Verified
Conference Paper
Open access
TLTony W Li
Tony W Li
AAArshia Arya
Arshia Arya
HJHaojian Jin
Haojian Jin
Software engineers’ unawareness of user feedback in earlier stages of design contributes to privacy issues in many products. Although extensive research exists on gathering and analyzing user feedback, there is limited understanding about how developers can integrate user feedback to improve product designs to better meet users’ privacy expectations. We use Zoom’s deprecated attendee attention tracking feature to explore issues with integrating user privacy feedback into software development, presenting public online critiques about this deprecated feature to 18 software engineers in semi-structured interviews and observing how they redesign this feature. Our results suggest that while integrating user feedback for privacy is potentially beneficial, it’s also fraught with challenges of polarized design suggestions, confirmation bias, and limited scope of perceived responsibility.

14 October 2024 in Proceedings of the ACM on Human-Computer Interaction
Verified
Journal Article
Open access
JWJinhe Wen
Jinhe Wen
ZZZhongyang Zhang
Zhongyang Zhang
TTTuan M. Tran
Tuan M. Tran
LMLianrui Mu
Lianrui Mu
TRTauhidur Rahman
Tauhidur Rahman
HJHaojian Jin
Haojian Jin
Regulations require video games to provide transparency regarding loot box odds to keep players informed, leading many games to disclose probabilities in various ways; yet, the extent of players' comprehension of loot box mechanics remains unclear. We performed a content analysis on 80 online posts to understand players' perceptions of loot box odds in two popular video games (Genshin Impact and Honkai: Star Rail). We then conducted semi-structured interviews with 24 players to explore the causes of these folk models across more games. Utilizing a bottom-up open coding approach, we created a taxonomy of folk models players have about loot boxes. We found that participants generally possessed inaccurate mental models of how loot boxes work, and they wanted game companies to enhance loot box transparency in three areas of probability disclosures: granularity, longitude, and scope.