This article proposes a three-step methodological framework called computational grounded theory, which combines expert human knowledge and hermeneutic skills with the processing power and pattern recognition of computers, producing a more methodologically rigorous but interpretive approach to content analysis. The first, pattern detection step, involves inductive computational exploration of text, using techniques such as unsupervised machine learning and word scores to help researchers to see novel patterns in their data. The second, pattern refinement step, returns to an interpretive engagement with the data through qualitative deep reading or further exploration of the data. The third, pattern confirmation step, assesses the inductively identified patterns using further computational and natural language processing techniques. The result is an efficient, rigorous, and fully reproducible computational grounded theory. This framework can be applied to any qualitative text as data, including transcribed speeches, interviews, open-ended survey data, or ethnographic field notes, and can address many potential research questions.

Advances in computer science and computational linguistics have yielded new, and faster, computational approaches to structuring and analyzing textual data. These approaches perform well on tasks like information extraction, but their ability to identify complex, socially constructed, and unsettled theoretical concepts—a central goal of sociological content analysis—has not been tested. To fill this gap, we compare the results produced by three common computer-assisted approaches—dictionary, supervised machine learning (SML), and unsupervised machine learning—to those produced through a rigorous hand-coding analysis of inequality in the news (N = 1,253 articles). Although we find that SML methods perform best in replicating hand-coded results, we document and clarify the strengths and weaknesses of each approach, including how they can complement one another. We argue that content analysts in the social sciences would do well to keep all these approaches in their toolkit, deploying them purposefully according to the task at hand.

Over the past decade, social scientists have adapted computational methods for qualitative text analysis, with the hope that they can match the accuracy and reliability of hand coding. The emergence of GPT and open-source generative large language models (LLMs) has transformed this process by shifting from programming to engaging with models using natural language, potentially mimicking the in-depth, inductive, and/or iterative process of qualitative analysis. We test the ability of generative LLMs to replicate and augment traditional qualitative coding, experimenting with multiple prompt structures across four closed- and open-source generative LLMs and proposing a workflow for conducting qualitative coding with generative LLMs. We find that LLMs can perform nearly as well as prior supervised machine learning models in accurately matching hand-coding output. Moreover, using generative LLMs as a natural language interlocutor closely replicates traditional qualitative methods, indicating their potential to transform the qualitative research process, despite ongoing challenges.

As the field of computational text analysis within the social sciences is maturing, computational methods are no longer seen as ends in themselves, but rather as means toward answering theoretically motivated research questions. The objective of this special issue is to showcase such research: the use of novel computational methods in the service of advancing substantive scientific knowledge. In presenting the contributions to the issue, we discuss several insights that emerge from this work, which hold relevance not only for current and aspiring practitioners of computational text analysis, but also for its skeptics. These concern the central role of theory in designing and executing computational research, the selection of appropriate techniques from a rapidly growing methodological toolkit, the benefits—and risks—of methodological bricolage, and the necessity of validating all aspects of the research process. The result is a set of broad considerations concerning the effective application of computational methods to substantive questions, illustrated by eight exemplary empirical studies.

Implicit bias is one of the most successful cases in recent memory of an academic concept being translated into practice. Its use in the National Science Foundation ADVANCE program—which seeks to promote gender equality in STEM (science, technology, engineering, mathematics) careers through institutional transformation—has raised fundamental questions about organizational change. How do advocates translate theories into practice? What makes some concepts more tractable than others? What happens to theories through this translation process? We explore these questions using the ADVANCE program as a case study. Using an inductive, theory-building approach and combination of computational and qualitative methods, we investigate how the concept of implicit bias was translated into practice through the ADVANCE program and identify five key features that made implicit bias useful as a change framework in the academic STEM setting. We find that the concept of implicit bias works programmatically because it is (1) demonstrable, (2) relatable, (3) versatile, (4) actionable, and (5) impartial. While enabling the concept’s diffusion, these characteristics also limit its scope. We reflect on implications for gender theories of organizational change and for practitioners.