Large language models (LLMs) can be used to estimate human attitudes and behavior, including measures of public opinion, a concept referred to as algorithmic fidelity. This study assesses the algorithmic fidelity and bias of LLMs in estimating public opinion about global warming. LLMs were conditioned on demographics and/or psychological covariates to simulate survey responses. Findings indicate that LLMs can effectively reproduce presidential voting behaviors but not global warming opinions unless the issue relevant covariates are included. When conditioned on both demographic and covariates, GPT-4 demonstrates improved accuracy, ranging from 53% to 91%, in predicting beliefs and attitudes about global warming. Additionally, we find an algorithmic bias that underestimates the global warming opinions of Black Americans. While highlighting the potential of LLMs to aid social science research, these results underscore the importance of conditioning, model selection, survey question format, and bias assessment when employing LLMs for survey simulation.
The present research explores how autonomous vehicle voice agent (AVVA) design influences autonomous vehicle passenger (AVP) intentions to adopt autonomous vehicles. An online experiment (N = 158) examined the role of gender stereotypes in response to an AVVA with respect to the technology acceptance model. The findings indicate that characteristics of the AVVA that are more consistent with the stereotypical expectation of the social role (informative male AVVA and social female AVVA) foster greater perceived ease of use (PEU) and perceived usefulness (PU) than inconsistent conditions (social male AVVA and informative female AVVA). The study offers theoretical implications regarding the technology acceptance model in the context of autonomous technologies as well as practical implications for the design of autonomous vehicle voice agents.
Beliefs and attitudes form the core of public opinion about climate change. Network analysis can reveal the structural configuration of these beliefs and attitudes. In this research, we utilize a belief system framework to identify key psychological elements, track change in the density of these belief systems over time and across political groups, and analyze the structural heterogeneity of belief systems within and between political groups in the United States. Drawing on fifteen waves of nationally representative survey data from 2010 to 2021 (N = 16,742), our findings indicate that worry about climate change is the most central psychological element. Interestingly, we find that among politically unaffiliated individuals, the connections between psychological elements have strengthened over time, implying an increase in the consistency of belief systems within this group. Despite the political polarization in beliefs about climate change between Republicans and Democrats, our findings reveal that the ways these two groups organize and structure climate change beliefs systems are not markedly different compared to those of other groups. These findings provide theoretical and practical insights for climate change experts and communicators.
Despite the popularity and efficiency of dictionary-based sentiment analysis (DSA) for public health research, limited empirical evidence has been produced about the validity of DSA and potential harms to the validity of DSA. A random sample of a second-hand Ebola tweet dataset was used to evaluate the validity of DSA compared to the manual coding approach and examine the influences of textual features on the validity of DSA. The results revealed substantial inconsistency between DSA and the manual coding approach. The presence of certain textual features such as negation can partially account for the inconsistency between DSA and manual coding. The findings imply that scholars should be careful and critical about findings in disease-related public health research that use DSA. Certain textual features should be more carefully addressed in DSA.
AI has the potential to enhance mental health by scaling support. However, its implementation brings uncertainties and challenges that require careful review to ensure safety. This study examined evolving public views on AI mental health support by analyzing relevant Reddit posts (n = 517). Following the release of ChatGPT in 2022, discussions about AI in the context of mental health surged, with a noticeable shift in preference toward large language models (LLMs) over conventional therapy chatbots. Users appreciated AI for its emotional support, companionship, and accessibility, while also expressing concerns about adverse effects and lack of conversational depth and emotional connection. Distinct patterns in how benefits and challenges were discussed emerged between experienced and non-experienced AI users, as well as between AI-focused and mental health-focused communities. AI-experienced users acknowledged both the benefits and limitations, whereas AI communities emphasized the positives and mental health communities highlighted the lack of conversational depth. These findings underscore the need for tailored communication strategies to set realistic expectations about the utility of AI in mental healthcare among different stakeholders. This research provides insights into developing ethical AI systems that complement traditional care while addressing current limitations.