Jonathan Stray

Mr. Jonathan Stray

orcid

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 July 2019 in Digital Journalism
Verified
Journal Article
Closed access
JSJonathan Stray
Jonathan Stray
Many have envisioned the use of AI methods to find hidden patterns of public interest in large volumes of data, greatly reducing the cost of investigative journalism. But so far only a few investigative stories have utilized AI methods, in relatively narrow ways. This paper surveys what has been accomplished in investigative reporting using AI techniques, why it has been difficult to apply more advanced methods, and what sorts of investigative journalism problems might be solved by AI in the near term. Journalism problems are often unique to a particular story, which means that training data is not readily available and the cost of complex models cannot be amortized over multiple projects. Much of the data relevant to a story is not publicly accessible but in the hands of governments and private entities, often requiring collection, negotiation, or purchase. Journalistic inference requires very high accuracy, or extensive manual checking, to avoid the risk of libel. The factors that make some set of facts “newsworthy” are deeply sociopolitical and therefore difficult to encode computationally. The biggest near-term potential for AI in investigative journalism lies in data preparation tasks, such as data extraction from diverse documents and probabilistic cross-database record linkage.

5 June 2024 in ACM Transactions on Recommender Systems
Verified
Journal Article
Open access
JSJonathan Stray
Jonathan Stray
AHAlon Halevy
Alon Halevy
PAParisa Assar
Parisa Assar
DHDylan Hadfield-Menell
Dylan Hadfield-Menell
CBCraig Boutilier
Craig Boutilier
NVNina Vasan
Nina Vasan
Recommender systems are the algorithms which select, filter, and personalize content across many of the world's largest platforms and apps. As such, their positive and negative effects on individuals and on societies have been extensively theorized and studied. Our overarching question is how to ensure that recommender systems enact the values of the individuals and societies that they serve. Addressing this question in a principled fashion requires technical knowledge of recommender design and operation, and also critically depends on insights from diverse fields including social science, ethics, economics, psychology, policy, and law. This article is a multidisciplinary effort to synthesize theory and practice from different perspectives, with the goal of providing a shared language, articulating current design approaches, and identifying open problems. We collect a set of values that seem most relevant to recommender systems operating across different domains, and then examine them from the perspectives of current industry practice, measurement, product design, and policy approaches. Important open problems include multi-stakeholder processes for defining values and resolving trade-offs, better values-driven measurements, recommender controls that people use, non-behavioral algorithmic feedback, optimization for long-term outcomes, causal inference of recommender effects, academic-industry research collaborations, and interdisciplinary policy-making.

4 November 2020 in International Journal of Community Well-Being
Verified
Journal Article
Open access
PDF file
JSJonathan Stray
Jonathan Stray
This paper investigates incorporating community well-being metrics into the objectives of optimization algorithms and the teams that build them. It documents two cases where a large platform appears to have modified their system to this end. Facebook incorporated “well-being” metrics in 2017, while YouTube began integrating “user satisfaction” metrics around 2015. Metrics tied to community well-being outcomes could also be used in many other systems, such as a news recommendation system that tries to increase exposure to diverse views, or a product recommendation system that opstimizes for the carbon footprint of purchased products. Generalizing from these examples and incorporating insights from participatory design and AI governance leads to a proposed process for integrating community well-being into commercial AI systems: identify and involve the affected community, choose a useful metric, use this metric as a managerial performance measure and/or an algorithmic objective, and evaluate and adapt to outcomes. Important open questions include the best approach to community participation and the uncertain business effects of this process.

18 April 2023 in Analytics
Verified
Journal Article
Open access
PDF file
JSJonathan Stray
Jonathan Stray
Researchers are starting to design AI-powered systems to automatically select and summarize the reports most relevant to each analyst, which raises the issue of bias in the information presented. This article focuses on the selection of relevant reports without an explicit query, a task known as recommendation. Drawing on previous work documenting the existence of human-machine feedback loops in recommender systems, this article reviews potential biases and mitigations in the context of intelligence analysis. Such loops can arise when behavioral “engagement” signals such as clicks or user ratings are used to infer the value of displayed information. Even worse, there can be feedback loops in the collection of intelligence information because users may also be responsible for tasking collection. Avoiding misalignment feedback loops requires an alternate, ongoing, non-engagement signal of information quality. Existing evaluation scales for intelligence product quality and rigor, such as the IC Rating Scale, could provide ground-truth feedback. This sparse data can be used in two ways: for human supervision of average performance and to build models that predict human survey ratings for use at recommendation time. Both techniques are widely used today by social media platforms. Open problems include the design of an ideal human evaluation method, the cost of skilled human labor, and the sparsity of the resulting data.

20 January 2023
Verified
Book Chapter
Closed access
PDF file
JSJonathan Stray
Jonathan Stray
This chapter proposes concrete methods for specifying the editorial behavior of news recommendation systems through collaboration between journalists and technologists to create metrics, data sets, feedback methods, and evaluation protocols. So far, the desired behavior of news recommenders has mostly been specified in terms of principles or guidelines. I argue that natural language specifications are inadequate because the translation to software must subsequently be undertaken by technical specialists, a process which requires consequential values-related decisions. Instead, I propose the specification of recommender editorial values through the collaborative creation of specific value-laden technical artifacts already used in contemporary engineering. These artifacts are much more precise than principles, yet do not require the technical understanding necessary to create novel algorithms.