Xi Chen, Ali Zeynali, Chico Camargo, Fabian Flöck, Devin Gaffney, Przemyslaw Grabowicz, Scott Hale, David Jurgens, Mattia Samory. Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). 2022.
Reddit, one of the largest global social media platforms, has undergone significant transformations since its inception in 2005. From a loosely structured, niche platform to a globally recognized company with a standardized and regulated governance system, Reddit’s evolution has been marked by a shift in the power dynamics between its owners, moderators, and users. 2015 and 2023 were marked by the occurrence of two prominent protests, termed “blackouts.” Moderators of numerous subreddits, though not all, disabled public access to their subreddits, thereby protesting the company’s policies and policy changes and challenging the company’s endeavors to exert further control over the platform. Drawing on Bourdieusian theory and relational methodology, we establish a computational social science approach to investigate the structural causes behind the two blackouts and contextualize the differences between them. We argue that these blackouts signify growing tensions within the socio-technical space of Reddit and an ongoing political, cultural, and economic reconfiguration of its power structure and political economy.
The disruptive offline mobilization of participants in online conspiracy theory (CT) discussions has highlighted the importance of understanding how online users may form radicalized conspiracy beliefs. While prior work researched the factors leading up to joining online CT discussions and provided theories of how conspiracy beliefs form, we have little understanding of how conspiracy radicalization evolves after users join CT discussion communities. In this paper, we provide the empirical modeling of various radicalization phases in online CT discussion participants. To unpack how conspiracy engagement is related to radicalization, we first characterize the users' journey through CT discussions via conspiracy engagement pathways. Specifically, by studying 36K Reddit users through their 169M contributions, we uncover four distinct pathways of conspiracy engagement: steady high, increasing, decreasing, and steady low. We further model three successive stages of radicalization guided by prior theoretical works. Specific sub-populations of users, namely those on steady high and increasing conspiracy engagement pathways, progress successively through various radicalization stages. In contrast, users on the decreasing engagement pathway show distinct behavior: they limit their CT discussions to specialized topics, participate in diverse discussion groups, and show reduced conformity with conspiracy subreddits. By examining users who disengage from online CT discussions, this paper provides promising insights about conspiracy recovery process.
Content validation, the assessment of whether measurement items adequately capture their intended constructs, is essential to scale development in organizational research but remains resource-intensive when conducted through conventional human-rating procedures. Natural language processing (NLP) offers a scalable complement for estimating definitional correspondence and distinctiveness, yet no study has systematically compared the full range of available approaches against direct human validity judgments across both multi-item and single-item measurement formats. The present study evaluates 16 NLP models spanning five architectural families (statistical models, static word embeddings, contextual sentence embeddings, task-specific interaction models and large language models) using two independent human-rated benchmark datasets. Results indicated that no single approach dominated across validity components or measurement formats, and that computational complexity was neither necessary nor sufficient for predictive accuracy. For multi-item correspondence, contextual sentence embeddings achieved top-group accuracy alongside supervised content-validation models; the choice of similarity metric substantially moderated performance for static embeddings. For multi-item distinctiveness, a broad upper group spanning all major architectural families achieved statistically indistinguishable accuracy. Single-item correspondence revealed a task-imposed performance ceiling with no architectural advantage. Practical recommendations are derived for each use case, prioritizing reproducibility, interpretability, and generalizability alongside predictive accuracy. A freely available web application implementing the recommended pipeline is introduced.