Kevyn Collins-Thompson

Dr. Kevyn Collins-Thompson

orcid

Research Focus & Metrics

Academic Career Length

18Years (2009-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 May 2017 in Information Retrieval Journal
Verified
Journal Article
Closed access
PDF file
RSRohail Syed
Rohail Syed
KCKevyn Collins-Thompson
Kevyn Collins-Thompson

1 August 2014 in Communications of the ACM
Verified
Journal Article
Open access
JTJaime Teevan
Jaime Teevan
KCKevyn Collins-Thompson
Kevyn Collins-Thompson
RWRyen W. White
Ryen W. White
SDSusan Dumais
Susan Dumais
Seeking to enrich the search experience by allowing for extra time and alternate resources.

1 January 2022 in Technology, Mind, and Behavior
Verified
Journal Article
Open access
PDF file
KMKathryn S McCarthy
Kathryn S McCarthy
SCScott A. Crossley
Scott A. Crossley
KMKayla Meyers
Kayla Meyers
UBUlrich Boser
Ulrich Boser
LALaura K. Allen
Laura K. Allen
MZMarcos Zampieri
Marcos Zampieri
The increased reliance on online education and educational technologies more generally has laid bare the need to more deeply consider how researchers, designers, and educators can improve the quality of technology-mediated learning. To address this need, more than two dozen experts from a variety of fields came together to discuss the challenges that educational technology must address in the immediate future. These experts were tasked with identifying barriers to and potential solutions for delivering high-quality and equitable online and remote education. This article examines the themes and topics that emerged from these discussions and proposes a Collaborative Framework for Accelerating Online Education. This framework highlights the need for rapid experimentation within larger design cycles as well as the coordination and cooperation of multiple stakeholders across all phases of research and development. The themes, topics, and framework that emerged from this work serve as a call to action for innovative approaches to developing and studying online education.

1 January 2014 in Proceedings of the American Society for Information Science and Technology
Verified
Journal Article
Open access
PDF file
SRSoo Young Rieh
Soo Young Rieh
JGJacek Gwizdka
Jacek Gwizdka
LFLuanne Freund
Luanne Freund
KCKevyn Collins-Thompson
Kevyn Collins-Thompson
There is growing recognition of the importance of learning as a search outcome and of the need to provide support for it. Yet, before we can consider learning as a part of search, we need to know how to assess it. This panel will focus on methods and measures for assessing learning in the context of search tasks and their outcomes. The panel will be interactive as the audience will be encouraged to engage in contributing their own experiences and ideas related to measures and methods to study learning as a part of search processes. Ideas and experiences with explicit and implicit indicators of learning and with evaluating learning outcomes will be shared during a dialogue between the audience and panelists. Outcomes from the panel discussions will contribute to formulating a research agenda for “search as learning.” The outcomes will be shared with the audience (and the wider ASIST community).

7 April 2016 in Journal of Medical Internet Research
Verified
Journal Article
Open access
TCTammy Chang
Tammy Chang
BVBianca A Verma
Bianca A Verma
TSTrevor Shull
Trevor Shull
MMMichelle Moniz
Michelle Moniz
LKLauren Kohatsu
Lauren Kohatsu
KCKevyn Collins-Thompson
Kevyn Collins-Thompson
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet

9 July 2024
Verified
Conference Paper
Open access
TATaimoor Arif
Taimoor Arif
SASumit Asthana
Sumit Asthana
KCKevyn Collins-Thompson
Kevyn Collins-Thompson
We present an empirical study evaluating the quality of multiple-choice questions (MCQs) generated by Large Language Models (LLMs) from a corpus of video transcripts of course lectures in an online data science degree program. With our database of thousands of generated questions, we conducted both human and automated judging of question quality on a representative sample using a broad set of criteria, including well-established Item Writing Flaw (IWF) categories. We found the number of average IWFs per MCQ ranged from 1.6 (rule-based verification) to 2.18 (LLM-based). Among the most frequently identified MCQ flaws were lack of enough context (17%) or answer choices with at least one implausible distractor (57%). Both human and automated assessment identified implausible distractors as one of the most frequent flaw categories. Results from our human annotation study were generally more positive (51--65% good items) compared to our automated assessment study results, which tended toward greater flaw identification (15--25% good items), depending on evaluation method.