Casey Canfield

Dr. Casey Canfield

orcid

Research Focus & Metrics

Academic Career Length

17Years (2010-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 September 2016 in Human Factors: The Journal of the Human Factors and Ergonomics Society
Verified
Journal Article
Closed access
CCCasey Canfield
Casey Canfield
BFBaruch Fischhoff
Baruch Fischhoff
ADAlex Davis
Alex Davis
We use signal detection theory to measure vulnerability to phishing attacks, including variation in performance across task conditions. Phishing attacks are difficult to prevent with technology alone, as long as technology is operated by people. Those responsible for managing security risks must understand user decision making in order to create and evaluate potential solutions. Using a scenario-based online task, we performed two experiments comparing performance on two tasks: detection, deciding whether an e-mail is phishing, and behavior, deciding what to do with an e-mail. In Experiment 1, we manipulated the order of the tasks and notification of the phishing base rate. In Experiment 2, we varied which task participants performed. In both experiments, despite exhibiting cautious behavior, participants’ limited detection ability left them vulnerable to phishing attacks. Greater sensitivity was positively correlated with confidence. Greater willingness to treat e-mails as legitimate was negatively correlated with perceived consequences from their actions and positively correlated with confidence. These patterns were robust across experimental conditions. Phishing-related decisions are sensitive to individuals’ detection ability, response bias, confidence, and perception of consequences. Performance differs when people evaluate messages or respond to them but not when their task varies in other ways. Based on these results, potential interventions include providing users with feedback on their abilities and information about the consequences of phishing, perhaps targeting those with the worst performance. Signal detection methods offer system operators quantitative assessments of the impacts of interventions and their residual vulnerability.

21 January 2024 in Artificial Intelligence in Medicine
Verified
Journal Article
Open access
HSHarishankar V. Subramanian
Harishankar V. Subramanian
CCCasey Canfield
Casey Canfield
DSDaniel B. Shank
Daniel B. Shank
The rise of complex AI systems in healthcare and other sectors has led to a growing area of research called Explainable AI (XAI) designed to increase transparency. In this area, quantitative and qualitative studies focus on improving user trust and task performance by providing system- and prediction-level XAI features. We analyze stakeholder engagement events (interviews and workshops) on the use of AI for kidney transplantation. From this we identify themes which we use to frame a scoping literature review on current XAI features. The stakeholder engagement process lasted over nine months covering three stakeholder group's workflows, determining where AI could intervene and assessing a mock XAI decision support system. Based on the stakeholder engagement, we identify four major themes relevant to designing XAI systems - 1) use of AI predictions, 2) information included in AI predictions, 3) personalization of AI predictions for individual differences, and 4) customizing AI predictions for specific cases. Using these themes, our scoping literature review finds that providing AI predictions before, during, or after decision-making could be beneficial depending on the complexity of the stakeholder's task. Additionally, expert stakeholders like surgeons prefer minimal to no XAI features, AI prediction, and uncertainty estimates for easy use cases. However, almost all stakeholders prefer to have optional XAI features to review when needed, especially in hard-to-predict cases. The literature also suggests that providing both system- and prediction-level information is necessary to build the user's mental model of the system appropriately. Although XAI features improve users' trust in the system, human-AI team performance is not always enhanced. Overall, stakeholders prefer to have agency over the XAI interface to control the level of information based on their needs and task complexity. We conclude with suggestions for future research, especially on customizing XAI features based on preferences and tasks.

13 January 2023 in Telecommunications Policy
Verified
Journal Article
Open access
JVJavier Valentín-Sívico
Javier Valentín-Sívico
CCCasey Canfield
Casey Canfield
SLSarah A. Low
Sarah A. Low
CGChristel Gollnick
Christel Gollnick
Having adequate access to the internet at home enhances quality-of-life for households and facilitates economic and social opportunities. Despite increased investment in response to the COVID-19 pandemic, millions of households in the rural United States still lack adequate access to high-speed internet. In this study, we evaluate a wireless broadband network deployed in Turney, a small, underserved rural community in northwest Missouri. In addition to collecting survey data before and after this internet intervention, we collected pre-treatment and post-treatment survey data from comparison communities to serve as a control group. Due to technical constraints, some of Turney's interested participants could not connect to the network, creating an additional comparison group. These comparisons suggest two primary findings, (1) changes in using the internet for employment, education, and health could not be directly attributed to the internet intervention, and (2) the internet intervention was associated with benefits stemming from the ability to use multiple devices at once. This study has implications for the design of future broadband evaluation studies, particularly those examining underserved rather than unserved communities. Recommendations for identifying appropriate outcome variables, executing recruitment strategies, and selecting the timing of surveys are made.

1 August 2013 in Applied Energy
Verified
Journal Article
Closed access
TKTamar Krishnamurti
Tamar Krishnamurti
ADAlexander L. Davis
Alexander L. Davis
GWGabrielle Wong-Parodi
Gabrielle Wong-Parodi
JWJack Wang
Jack Wang
CCCasey Canfield
Casey Canfield

24 February 2022 in Human Factors: The Journal of the Human Factors and Ergonomics Society
Verified
Journal Article
Open access
RKRyan Kenny
Ryan Kenny
BFBaruch Fischhoff
Baruch Fischhoff
ADAlex Davis
Alex Davis
KCKathleen M. Carley
Kathleen M. Carley
CCCasey Canfield
Casey Canfield
Objective: We examine individuals’ ability to detect social bots among Twitter personas, along with participant and persona features associated with that ability. Background: Social media users need to distinguish bots from human users. We develop and demonstrate a methodology for assessing those abilities, with a simulated social media task. Method: We analyze performance from a signal detection theory perspective, using a task that asked lay participants whether each of 50 Twitter personas was a human or social bot. We used the agreement of two machine learning models to estimate the probability of each persona being a bot. We estimated the probability of participants indicating that a persona was a bot with a generalized linear mixed-effects model using participant characteristics (social media experience, analytical reasoning, and political views) and stimulus characteristics (bot indicator score and political tone) as regressors. Results: On average, participants had modest sensitivity (d’) and a criterion that favored responding “human.” Exploratory analyses found greater sensitivity for participants (a) with less self-reported social media experience, (b) greater analytical reasoning ability, and (c) who were evaluating personas with opposing political views. Some patterns varied with participants' political identity. Conclusions: Individuals have limited ability to detect social bots, with greater aversion to mistaking bots for humans than vice versa. Greater social media experience and myside bias appeared to reduce performance, as did less analytical reasoning ability. Application: These patterns suggest the need for interventions, especially when users feel most familiar with social media.

21 May 2022 in Human Factors: The Journal of the Human Factors and Ergonomics Society
Verified
Journal Article
Closed access
HEHannah Elder
Hannah Elder
CCCasey Canfield
Casey Canfield
DSDaniel B. Shank
Daniel B. Shank
TRTobias Rieger
Tobias Rieger
CHCasey Hines
Casey Hines
Objective: This study manipulates the presence and reliability of AI recommendations for risky decisions to measure the effect on task performance, behavioral consequences of trust, and deviation from a probability matching collaborative decision-making model. Background: Although AI decision support improves performance, people tend to underutilize AI recommendations, particularly when outcomes are uncertain. As AI reliability increases, task performance improves, largely due to higher rates of compliance (following action recommendations) and reliance (following no-action recommendations). Methods: In a between-subject design, participants were assigned to a high reliability AI, low reliability AI, or a control condition. Participants decided whether to bet that their team would win in a series of basketball games tying compensation to performance. We evaluated task performance (in accuracy and signal detection terms) and the behavioral consequences of trust (via compliance and reliance). Results: AI recommendations improved task performance, had limited impact on risk-taking behavior, and were under-valued by participants. Accuracy, sensitivity ( d’ ), and reliance increased in the high reliability AI condition, but there was no effect on response bias ( c ) or compliance. Participant behavior was only consistent with a probability matching model for compliance in the low reliability condition. Conclusion: In a pay-off structure that incentivized risk-taking, the primary value of the AI recommendations was in determining when to perform no action (i.e., pass on bets). Application: In risky contexts, designers need to consider whether action or no-action recommendations will be more influential to design appropriate interventions.