Jeffrey Stewart

Dr. Jeffrey Stewart

orcid

Research Focus & Metrics

Academic Career Length

16Years (2011-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 January 2020 in Language Testing
Verified
Journal Article
Closed access
PDF file
SMStuart McLean
Stuart McLean
JSJeffrey Stewart
Jeffrey Stewart
ABAaron Olaf Batty
Aaron Olaf Batty
Vocabulary’s relationship to reading proficiency is frequently cited as a justification for the assessment of L2 written receptive vocabulary knowledge. However, to date, there has been relatively little research regarding which modalities of vocabulary knowledge have the strongest correlations to reading proficiency, and observed differences have often been statistically non-significant. The present research employs a bootstrapping approach to reach a clearer understanding of relationships between various modalities of vocabulary knowledge to reading proficiency. Test-takers (N = 103) answered 1000 vocabulary test items spanning the third 1000 most frequent English words in the New General Service List corpus (Browne, Culligan, & Phillips, 2013). Items were answered under four modalities: Yes/No checklists, form recall, meaning recall, and meaning recognition. These pools of test items were then sampled with replacement to create 1000 simulated tests ranging in length from five to 200 items and the results were correlated to the Test of English for International Communication (TOEIC®) Reading scores. For all examined test lengths, meaning-recall vocabulary tests had the highest average correlations to reading proficiency, followed by form-recall vocabulary tests. The results indicated that tests of vocabulary recall are stronger predictors of reading proficiency than tests of vocabulary recognition, despite the theoretically closer relationship of vocabulary recognition to reading.

23 December 2020 in Language Testing
Verified
Journal Article
Open access
PDF file
Henrik GyllstadHenrik Gyllstad
Henrik Gyllstad
SMStuart McLean
Stuart McLean
JSJeffrey Stewart
Jeffrey Stewart
The last three decades have seen an increase of tests aimed at measuring an individual’s vocabulary level or size. The target words used in these tests are typically sampled from word frequency lists, which are in turn based on language corpora. Conventionally, test developers sample items from frequency bands of 1000 words; different tests employ different sampling ratios. Some have as few as 5 or 10 items representing the underlying population of words, whereas other tests feature a larger number of items, such as 24, 30, or 40. However, very rarely are the sampling size choices supported by clear empirical evidence. Here, using a bootstrapping approach, we illustrate the effect that a sample-size increase has on confidence intervals of individual learner vocabulary knowledge estimates, and on the inferences that can safely be made from test scores. We draw on a unique dataset consisting of adult L1 Japanese test takers’ performance on two English vocabulary test formats, each featuring 1000 words. Our analysis shows that there are few purposes and settings where as few as 5 to 10 sampled items from a 1000-word frequency band (1K) are sufficient. The use of 30 or more items per 1000-word frequency band and tests consisting of fewer bands is recommended.

1 June 2011 in TESOL Quarterly
Verified
Journal Article
Closed access
JSJeffrey Stewart
Jeffrey Stewart
DWDavid A. White
David A. White

3 July 2014 in Language Assessment Quarterly
Verified
Journal Article
Closed access
JSJeffrey Stewart
Jeffrey Stewart
Validated under a Rasch framework (Beglar, 2010), the Vocabulary Size Test (VST) (Nation & Beglar, 2007) is an increasingly popular measure of decontextualized written receptive vocabulary size in the field of second language acquisition. However, although the validation indicates that the test has high internal reliability, still unaddressed is the possibility that it overestimates learner vocabulary size due to guessing effects inherent in its multiple-choice format, as size estimates are made by multiplying its raw score by a constant (100 or 200). This article argues that the VST’s multiple-choice format results in a test of passive recognition of words that does not approximate the experience of readers of authentic English texts, details drawbacks of the Rasch framework and mean-square fit statistics in detecting the overall contribution of guessing effects to raw test scores that could have allowed such deficiencies to remain undetected during the test’s validation, overviews challenges that multiple-choice formats pose for vocabulary tests, and concludes by proposing methods of testing and analysis that can address these concerns.

14 June 2012 in TESOL Quarterly
Verified
Journal Article
Closed access
PDF file
JSJeffrey Stewart
Jeffrey Stewart
ABAaron Olaf Batty
Aaron Olaf Batty
NBNicholas Bovee
Nicholas Bovee

24 April 2023 in Language Testing
Verified
Journal Article
Open access
JSJeffrey Stewart
Jeffrey Stewart
Henrik GyllstadHenrik Gyllstad
Henrik Gyllstad
CNChristopher Nicklin
Christopher Nicklin
SMStuart McLean
Stuart McLean
The purpose of this paper is to (a) establish whether meaning recall and meaning recognition item formats test psychometrically distinct constructs of vocabulary knowledge which measure separate skills, and, if so, (b) determine whether each construct possesses unique properties predictive of L2 reading proficiency. Factor analyses and hierarchical regression were conducted on results derived from the two vocabulary item formats in order to test this hypothesis. The results indicated that although the two-factor model had better fit and meaning recall and meaning recognition can be considered distinct psychometrically, discriminant validity between the two factors is questionable. In hierarchical regression models, meaning recognition knowledge did not make a statistically significant contribution to explaining reading proficiency over meaning recall knowledge. However, when the roles were reversed, meaning recall did make a significant contribution to the model beyond the variance explained by meaning recognition alone. The results suggest that meaning recognition does not tap into unique aspects of vocabulary knowledge and provide empirical support for meaning recall as a superior predictor of reading proficiency for research purposes.