Xi Chen, Ali Zeynali, Chico Camargo, Fabian Flöck, Devin Gaffney, Przemyslaw Grabowicz, Scott Hale, David Jurgens, Mattia Samory. Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). 2022.
With this work, we present a publicly available data set of the history of all the references (more than 55 million) ever used in the English Wikipedia until June 2019. We have applied a new method for identifying and monitoring references in Wikipedia, so that for each reference we can provide data about associated actions: creation, modifications, deletions, and reinsertions. The high accuracy of this method and the resulting data set was confirmed via a comprehensive crowdworker labeling campaign. We use the data set to study the temporal evolution of Wikipedia references as well as users’ editing behavior. We find evidence of a mostly productive and continuous effort to improve the quality of references: There is a persistent increase of reference and document identifiers (DOI, PubMedID, PMC, ISBN, ISSN, ArXiv ID) and most of the reference curation work is done by registered humans (not bots or anonymous editors). We conclude that the evolution of Wikipedia references, including the dynamics of the community processes that tend to them, should be leveraged in the design of relevance indexes for altmetrics, and our data set can be pivotal for such an effort.
The revision histories of Wikipedia articles are a rich source of data about the interactions of editors with each other and with the content, yet they are not straightforward to mine or understand. We describe two tools for visual analytics that support this effort: (i) An interactive browser extension to study word authorship, age, and conflict dynamics, which provides an overlay on live Wikipedia articles; and (ii) a novel interactive Jupyter Notebook package that allows us to run analyses of editorial dynamics out-of-the-box and is easily modifiable. Both leverage live data for any article on demand from several Web APIs, centering on our own WikiWho service, providing the most accurate mining of live word-level changes currently available. We show how these tools enable the exploration of the survival of content, productivity of editors, conflict dynamics, and other metrics through low-barrier interfaces while providing the opportunity for more quantitative investigations via access to the notebooks’ underlying data structures.