Nancy A. Obuchowski

Nancy A. Obuchowski

Auto-generated profile, based on publicly available information

Nancy A. Obuchowski

Department of Quantitative Health Sciences and Department of Diagnostic Radiology, Cleveland Clinic Foundation, Cleveland OH, United States

Research Focus & Metrics

Academic Career Length

25Years (2002-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 March 2018 in Physics in Medicine & Biology
Verified
Journal Article
Closed access
PDF file
NONancy A Obuchowski
Nancy A Obuchowski
JBJennifer A Bullen
Jennifer A Bullen
Receiver operating characteristic (ROC) analysis is a tool used to describe the discrimination accuracy of a diagnostic test or prediction model. While sensitivity and specificity are the basic metrics of accuracy, they have many limitations when characterizing test accuracy, particularly when comparing the accuracies of competing tests. In this article we review the basic study design features of ROC studies, illustrate sample size calculations, present statistical methods for measuring and comparing accuracy, and highlight commonly used ROC software. We include descriptions of multi-reader ROC study design and analysis, address frequently seen problems of verification and location bias, discuss clustered data, and provide strategies for testing endpoints in ROC studies. The methods are illustrated with a study of transmission ultrasound for diagnosing breast lesions.

17 January 2024 in Insights into Imaging
Verified
Journal Article
Open access
PDF file
BKBurak Kocak
Burak Kocak
TDTugba Akinci D'Antonoli
Tugba Akinci D'Antonoli
NMNathaniel Mercaldo
Nathaniel Mercaldo
AAAngel Alberich-Bayarri
Angel Alberich-Bayarri
BBBettina Baeßler
Bettina Baeßler
Renato CuocoloRenato Cuocolo
Renato Cuocolo
Purpose: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies. Methods: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated. Result: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community. Conclusion: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers. Critical relevance statement: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning. Key points: • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score (https://metricsscore.github.io/metrics/METRICS.html) and a repository created to collect feedback from the radiomics community (https://github.com/metricsscore/metrics). Graphical

11 June 2014 in Statistical Methods in Medical Research
Verified
Journal Article
Closed access
PDF file
NONancy A Obuchowski
Nancy A Obuchowski
ARAnthony P Reeves
Anthony P Reeves
EHErich P Huang
Erich P Huang
XWXiao-Feng Wang
Xiao-Feng Wang
ABAndrew Buckler
Andrew Buckler
(G(for the Algorithm Comparison Working Group)
(for the Algorithm Comparison Working Group)
Quantitative biomarkers from medical images are becoming important tools for clinical diagnosis, staging, monitoring, treatment planning, and development of new therapies. While there is a rich history of the development of quantitative imaging biomarker (QIB) techniques, little attention has been paid to the validation and comparison of the computer algorithms that implement the QIB measurements. In this paper we provide a framework for QIB algorithm comparisons. We first review and compare various study designs, including designs with the true value (e.g. phantoms, digital reference images, and zero-change studies), designs with a reference standard (e.g. studies testing equivalence with a reference standard), and designs without a reference standard (e.g. agreement studies and studies of algorithm precision). The statistical methods for comparing QIB algorithms are then presented for various study types using both aggregate and disaggregate approaches. We propose a series of steps for establishing the performance of a QIB algorithm, identify limitations in the current statistical literature, and suggest future directions for research.

1 August 2014 in Epilepsy & Behavior
Verified
Journal Article
Closed access
JRJaivir S. Rathore
Jaivir S. Rathore
LJLara E. Jehi
Lara E. Jehi
YFYouran Fan
Youran Fan
SPSima I. Patel
Sima I. Patel
NFNancy Foldvary-Schaefer
Nancy Foldvary-Schaefer
GTGeorge E. Tesar
George E. Tesar

1 April 2022 in Radiology
Verified
Journal Article
Closed access
NONancy A. Obuchowski
Nancy A. Obuchowski
JBJennifer Bullen
Jennifer Bullen
The multireader multicase (MRMC) design is a commonly used tool in diagnostic imaging, yet if poorly designed or misanalyzed, MRMC studies can provide misleading results and waste valuable resources. The design and analysis of multireader multicase (MRMC) studies are quite challenging. These studies differ from most medical studies because they need a reference standard and sampling from two po...

7 January 2020 in Radiology
Verified
Journal Article
Closed access
PKPaul E Kinahan
Paul E Kinahan
EPEric S. Perlman
Eric S. Perlman
JSJohn J. Sunderland
John J. Sunderland
RSRathan Subramaniam
Rathan Subramaniam
SWScott D. Wollenweber
Scott D. Wollenweber
RWRichard Wahl
Richard Wahl
This article provides a summary of the claims of precision of standardized uptake value measurements and the context of the technically confirmed fluorodeoxyglucose PET/CT Profile by the Quantitati... The Quantitative Imaging Biomarkers Alliance (QIBA) Profile for fluorodeoxyglucose (FDG) PET/CT imaging was created by QIBA to both characterize and reduce the variability of standardized uptake va...