Lane Harrison

Lane Harrison

Auto-generated profile, based on publicly available information

Lane Harrison

Worcester Polytechnic Institute

Research Focus & Metrics

Academic Career Length

13Years (2014-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 October 2023 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
YCYuan Cui
Yuan Cui
LGLily W. Ge
Lily W. Ge
YDYiren Ding
Yiren Ding
FYFumeng Yang
Fumeng Yang
LHLane Harrison
Lane Harrison
MKMatthew Kay
Matthew Kay
Visualization literacy is an essential skill for accurately interpreting data to inform critical decisions. Consequently, it is vital to understand the evolution of this ability and devise targeted interventions to enhance it, requiring concise and repeatable assessments of visualization literacy for individuals. However, current assessments, such as the Visualization Literacy Assessment Test ( vlat ), are time-consuming due to their fixed, lengthy format. To address this limitation, we develop two streamlined computerized adaptive tests ( cats ) for visualization literacy, a-vlat and a-calvi , which measure the same set of skills as their original versions in half the number of questions. Specifically, we (1) employ item response theory (IRT) and non-psychometric constraints to construct adaptive versions of the assessments, (2) finalize the configurations of adaptation through simulation, (3) refine the composition of test items of a-calvi via a qualitative study, and (4) demonstrate the test-retest reliability (ICC: 0.98 and 0.98) and convergent validity (correlation: 0.81 and 0.66) of both CATS via four online studies. We discuss practical recommendations for using our CATS and opportunities for further customization to leverage the full potential of adaptive assessments. All supplemental materials are available at https://osf.io/a6258/ .

10 September 2024 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
YCYuan Cui
Yuan Cui
LGLily W. Ge
Lily W. Ge
YDYiren Ding
Yiren Ding
LHLane Harrison
Lane Harrison
FYFumeng Yang
Fumeng Yang
MKMatthew Kay
Matthew Kay
Visualization items—factual questions about visualizations that ask viewers to accomplish visualization tasks-are regularly used in the field of information visualization as educational and evaluative materials. For example, researchers of visualization literacy require large, diverse banks of items to conduct studies where the same skill is measured repeatedly on the same participants. Yet, generating a large number of high-quality, diverse items requires significant time and expertise. To address the critical need for a large number of diverse visualization items in education and research, this paper investigates the potential for large language models (LLMS) to automate the generation of multiple-choice visualization items. Through an iterative design process, we develop the VILA (Visualization Items Generated by Large LAnguage Models) pipeline, for efficiently generating visualization items that measure people's ability to accomplish visualization tasks. We use the VILA pipeline to generate 1,404 candidate items across 12 chart types and 13 visualization tasks. In collaboration with 11 visualization experts, we develop an evaluation rulebook which we then use to rate the quality of all candidate items. The result is the VILA bank of ~1, 100 items. From this evaluation, we also identify and classify current limitations of the VILA pipeline, and discuss the role of human oversight in ensuring quality. In addition, we demonstrate an application of our work by creating a visualization literacy test, VILA-VLAT, which measures people's ability to complete a diverse set of tasks on various types of visualizations; comparing it to the existing VLAT, VILA-VLAT shows moderate to high convergent validity (R = 0.70). Lastly, we discuss the application areas of the VILA pipeline and the VILA bank and provide practical recommendations for their use. All supplemental materials are available at https://osf.io/ysrhq/.

9 December 2022 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
RDRussell Davis
Russell Davis
XPXiaoying Pu
Xiaoying Pu
YDYiren Ding
Yiren Ding
BHBrian D. Hall
Brian D. Hall
KBKaren Bonilla
Karen Bonilla
LHLane Harrison
Lane Harrison
Graphical perception studies typically measure visualization encoding effectiveness using the error of an “average observer”, leading to canonical rankings of encodings for numerical attributes: e.g. , position $>$ area $>$ angle $>$ volume. Yet different people may vary in their ability to read different visualization types, leading to variance in this ranking across individuals not captured by population-level metrics using “average observer” models. One way we can bridge this gap is by recasting classic visual perception tasks as tools for assessing individual performance, in addition to overall visualization performance. In this paper we replicate and extend Cleveland and McGill's graphical comparison experiment using Bayesian multilevel regression, using these models to explore individual differences in visualization skill from multiple perspectives. The results from experiments and modeling indicate that some people show patterns of accuracy that credibly deviate from the canonical rankings of visualization effectiveness. We discuss implications of these findings, such as a need for new ways to communicate visualization effectiveness to designers, how patterns in individuals' responses may show systematic biases and strategies in visualization judgment, and how recasting classic visual perception tasks as tools for assessing individual performance may offer new ways to quantify aspects of visualization literacy. Experiment data, source code, and analysis scripts are available at the following repository: https://osf.io/8ub7t/?view_only=9be4798797404a4397be3c6fc2a68cc0 .

19 April 2023
Verified
Conference Paper
Open access
FYFumeng Yang
Fumeng Yang
YMYuxin Ma
Yuxin Ma
LHLane Harrison
Lane Harrison
JTJames Tompkin
James Tompkin
DLDavid H. Laidlaw
David H. Laidlaw
How deep neural networks can aid visualization perception research is a wide-open question. This paper provides insights from three perspectives—prediction, generalization, and interpretation—via training and analyzing deep convolutional neural networks on human correlation judgments in scatterplots across three studies. The first study assesses the accuracy of twenty-nine neural network architectures in predicting human judgments, finding that a subset of the architectures (e.g., VGG-19) has comparable accuracy to the best-performing regression analyses in prior research. The second study shows that the resulting models from the first study display better generalizability than prior models on two other judgment datasets for different scatterplot designs. The third study interprets visual features learned by a convolutional neural network model, providing insights about how the model makes predictions, and identifies potential features that could be investigated in human correlation perception studies. Together, this paper suggests that deep neural networks can serve as a tool for visualization perception researchers in devising potential empirical study designs and hypothesizing about perpetual judgments. The preprint, data, code, and training logs are available at https://doi.org/10.17605/osf.io/exa8m.

11 May 2024
Verified
Conference Paper
Open access
LGLily W. Ge
Lily W. Ge
MHMaryam Hedayati
Maryam Hedayati
YCYuan Cui
Yuan Cui
YDYiren Ding
Yiren Ding
KBKaren Bonilla
Karen Bonilla
MKMatthew Kay
Matthew Kay
Researchers have proposed many definitions of visualization literacy, targeting various aspects of the term. But we have yet to fully capture what it really means to be literate in visualizations, which has important downstream implications, such as how to effectively teach visualization skills to younger generations. We ran a meetup at IEEE VIS 2022 that attracted over 30 researchers in the field, who discussed aspects of visualization literacy such as how we measure it, how we can improve it, how it develops, and how it relates to other literacies. ACM CHI has a track record of attracting researchers from various fields such as visualization, learning sciences, and design, advancing research through both quantitative and qualitative approaches in and around HCI. For this year’s CHI, we propose to run a one-day workshop with the goal of further developing actionable research agendas to more comprehensively define, understand, and improve visualization literacy. By continuing critical discussions with diverse perspectives from the CHI community, we can deepen investigations of visualization literacy through multiple lenses, such as measurement, interventions, and pedagogy.

3 June 2024
Verified
Conference Paper
Open access
MAMallak Alkhathlan
Mallak Alkhathlan
KCKathleen Cachel
Kathleen Cachel
HSHilson Shrestha
Hilson Shrestha
LHLane Harrison
Lane Harrison
ERElke Rundensteiner
Elke Rundensteiner
Algorithmic decision-making using rankings— prevalent in areas from hiring and bail to university admissions— raises concerns of potential bias. In this paper, we explore the alignment between people’s perceptions of fairness and two popular fairness metrics designed for rankings. In a crowdsourced experiment with 480 participants, people rated the perceived fairness of a hypothetical scholarship distribution scenario. Results suggest a strong inclination towards relying on explicit score values. There is also evidence of people’s preference for one fairness metric, NDKL, over the other metric, ARP. Qualitative results paint a more complex picture: some participants endorse meritocratic award schemes and express concerns about fairness metrics being used to modify rankings; while other participants acknowledge socio-economic factors in score-based rankings as justification for adjusting rankings. In summary, we find that operationalizing algorithmic fairness in practice is a balancing act between mitigating harms towards marginalized groups and societal conventions of leveraging traditional performance scores such as grades in decision-making contexts.