John Norcini

John Norcini

Auto-generated profile, based on publicly available information

John Norcini

SUNY Upstate Medical University Syracuse Pennsylvania USA

Research Focus & Metrics

Academic Career Length

24Years (2003-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2007 in Medical Teacher
Verified
Journal Article
Closed access
JNJohn Norcini
John Norcini
VBVanessa Burch
Vanessa Burch
There has been concern that trainees are seldom observed, assessed, and given feedback during their workplace-based education. This has led to an increasing interest in a variety of formative assessment methods that require observation and offer the opportunity for feedback. To review some of the literature on the efficacy and prevalence of formative feedback, describe the common formative assessment methods, characterize the nature of feedback, examine the effect of faculty development on its quality, and summarize the challenges still faced. The research literature on formative assessment and feedback suggests that it is a powerful means for changing the behaviour of trainees. Several methods for assessing it have been developed and there is preliminary evidence of their reliability and validity. A variety of factors enhance the efficacy of workplace-based assessment including the provision of feedback that is consistent with the needs of the learner and focused on important aspects of the performance. Faculty plays a critical role and successful implementation requires that they receive training. There is a need for formative assessment which offers trainees the opportunity for feedback. Several good methods exist and feedback has been shown to have a major influence on learning. The critical role of faculty is highlighted, as is the need for strategies to enhance their participation and training.

18 March 2003 in Annals of Internal Medicine
Verified
Journal Article
Closed access
PDF file
JNJohn J. Norcini
John J. Norcini
LBLinda L. Blank
Linda L. Blank
FDF Daniel Duffy
F Daniel Duffy
GFGregory S. Fortna
Gregory S. Fortna
To evaluate the miniclinical evaluation exercise (mini-CEX), which assesses the clinical skills of residents. Observational study and psychometric assessment of the mini-CEX. 21 internal medicine training programs. Data from 1228 mini-CEX encounters involving 421 residents and 316 evaluators. The encounters were assessed for the type of visit, sex and complexity of the patient, when the encounter occurred, length of the encounter, ratings provided, and the satisfaction of the examiners. Using this information, we determined the overall average ratings for residents in all categories, the reliability of the mini-CEX scores, and the effects of the characteristics of the patients and encounters. Interviewing skills, physical examination, professionalism, clinical judgment, counseling, organization and efficiency, and overall competence were evaluated. Residents were assessed in various clinical settings with a diverse set of patient problems. Residents received the lowest ratings in the physical examination and the highest ratings in professionalism. Comparisons over the first year of training showed statistically significant improvement in all aspects of competence, and the method generated reliable ratings. The measurement characteristics of the mini-CEX are similar to those of other performance assessments, such as standardized patients. Unlike these assessments, the difficulty of the examination will vary with the patients that a resident encounters. This effect is mitigated to a degree by the examiners, who slightly overcompensate for patient difficulty, and by the fact that each resident interacts with several patients. Furthermore, the mini-CEX has higher fidelity than these formats, permits evaluation based on a much broader set of clinical settings and patient problems, and is administered on site.

23 February 2011 in Medical Teacher
Verified
Journal Article
Closed access
PDF file
JNJohn Norcini
John Norcini
BABrownell Anderson
Brownell Anderson
VBValdes Roberto Bollela
Valdes Roberto Bollela
VBVanessa Burch
Vanessa Burch
MCManuel João Costa
Manuel João Costa
TRTrudie Roberts
Trudie Roberts
In this article, we outline criteria for good assessment that include: (1) validity or coherence, (2) reproducibility or consistency, (3) equivalence, (4) feasibility, (5) educational effect, (6) catalytic effect, and (7) acceptability. Many of the criteria have been described before and we continue to support their importance here. However, we place particular emphasis on the catalytic effect of the assessment, which is whether the assessment provides results and feedback in a fashion that creates, enhances, and supports education. These criteria do not apply equally well to all situations. Consequently, we discuss how the purpose of the test (summative versus formative) and the perspectives of stakeholders (examinees, patients, teachers-educational institutions, healthcare system, and regulators) influence the importance of the criteria. Finally, we offer a series of practice points as well as next steps that should be taken with the criteria. Specifically, we recommend that the criteria be expanded or modified to take account of: (1) the perspectives of patients and the public, (2) the intimate relationship between assessment, feedback, and continued learning, (3) systems of assessment, and (4) accreditation systems.

9 October 2018 in Medical Teacher
Verified
Journal Article
Closed access
JNJohn Norcini
John Norcini
MAM. Brownell Anderson
M. Brownell Anderson
VBValdes Bollela
Valdes Bollela
VBVanessa Burch
Vanessa Burch
MCManuel João Costa
Manuel João Costa
DSDavid Swanson
David Swanson
Introduction: In 2010, the Ottawa Conference produced a set of consensus criteria for good assessment. These were well received and since then the working group monitored their use. As part of the 2010 report, it was recommended that consideration be given in the future to preparing similar criteria for systems of assessment. Recent developments in the field suggest that it would be timely to undertake that task and so the working group was reconvened, with changes in membership to reflect broad global representation.Methods: Consideration was given to whether the initially proposed criteria continued to be appropriate for single assessments and the group believed that they were. Consequently, we reiterate the criteria that apply to individual assessments and duplicate relevant portions of the 2010 report.Results and discussion: This paper also presents a new set of criteria that apply to systems of assessment and, recognizing the challenges of implementation, offers several issues for further consideration. Among these issues are the increasing diversity of candidates and programs, the importance of legal defensibility in high stakes assessments, globalization and the interest in portable recognition of medical training, and the interest among employers and patients in how medical education is delivered and how progression decisions are made.

1 August 2010 in Health Affairs
Verified
Journal Article
Open access
PDF file
JNJohn J. Norcini
John J. Norcini
JBJohn R. Boulet
John R. Boulet
WDW. Dale Dauphinee
W. Dale Dauphinee
AOAmy Opalek
Amy Opalek
IKIan D. Krantz
Ian D. Krantz
SASuzanne T. Anderson
Suzanne T. Anderson
One-quarter of practicing physicians in the United States are graduates of international medical schools. The quality of care provided by doctors educated abroad has been the subject of ongoing concern. Our analysis of 244,153 hospitalizations in Pennsylvania found that patients of doctors who graduated from international medical schools and were not U.S. citizens at the time they entered medical school had significantly lower mortality rates than patients cared for by doctors who graduated from U.S. medical schools or who were U.S. citizens and received their degrees abroad. The patient population consisted of those with congestive heart failure or acute myocardial infarction. We found no significant mortality difference when comparing all international medical graduates with all U.S. medical school graduates.

5 May 2005 in Medical Education
Verified
Journal Article
Closed access
PDF file
JNJohn J Norcini
John J Norcini