Benjamin Kremmel

Benjamin Kremmel

Auto-generated profile, based on publicly available information

Benjamin Kremmel

University of Innsbruck, Austria

Research Focus & Metrics

Academic Career Length

14Years (2013-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 November 2019 in Language Assessment Quarterly
Verified
Journal Article
Open access
BKBenjamin Kremmel
Benjamin Kremmel
LHLuke Harding
Luke Harding
While scholars have proposed different models of language assessment literacy (LAL), these models have mostly comprised prescribed sets of components based on principles of good practice. As such, these models remain theoretical in nature, and represent the perspectives of language assessment researchers rather than stakeholders themselves. The project from which the current study is drawn was designed to address this issue through an empirical investigation of the LAL needs of different stakeholder groups. Central to this aim was the development of a rigorous and comprehensive survey which would illuminate the dimensionality of LAL and generate profiles of needs across these dimensions. This paper reports on the development of an instrument designed for this purpose: the Language Assessment Literacy Survey. We first describe the expert review and pretesting stages of survey development. Then we report on the results of an exploratory factor analysis based on data from a large-scale administration (N = 1086), where respondents from a range of stakeholder groups across the world judged the LAL needs of their peers. Finally, selected results from the large-scale administration are presented to illustrate the survey’s utility, specifically comparing the responses of language teachers, language testing/assessment developers and language testing/assessment researchers.

1 October 2016 in Language Assessment Quarterly
Verified
Journal Article
Closed access
BKBenjamin Kremmel
Benjamin Kremmel
NSNorbert Schmitt
Norbert Schmitt
The scores from vocabulary size tests have typically been interpreted as demonstrating that the target words are “known” or “learned.” But “knowing” a word should entail the ability to use it in real language communication in one or more of the four skills. It should also entail deeper knowledge, such as knowing the word’s derivative forms or collocations. However, little is known about how informative various vocabulary item formats are about lexical employability or mastery of aspects of word knowledge. That is, we have little idea about what score interpretations are warranted from various item formats. This article reports on two initial studies that investigated four form-meaning item formats (multiple matching, multiple-choice, and two types of cloze) to determine how informative they can be about lexical employability and knowledge of derivatives and collocations. Results showed that the four formats performed poorly and that it may not be warranted to interpret scores from these formats as showing that vocabulary is known to the level where it can be used when reading. Furthermore, the findings suggest that scores from these tests cannot be taken to mean that the words are known to any deeper degree than just the form-meaning link.

16 July 2020 in Language Testing
Verified
Journal Article
Closed access
PDF file
DIDaniel Isbell
Daniel Isbell
BKBenjamin Kremmel
Benjamin Kremmel
Administration of high-stakes language proficiency tests has been disrupted in many parts of the world as a result of the 2019 novel coronavirus pandemic. Institutions that rely on test scores have been forced to adapt, and in many cases this means using scores from a different test, or a new online version of an existing test, that can be taken at home. The switch to accepting at-home proficiency tests for high-stakes decisions raises many concerns for stakeholders, such as technological demands, exam security, and validity of score use. Along these lines, this thematic review addresses such concerns and features brief reviews of seven options in at-home proficiency testing: ACTFL Assessments, Duolingo English Test, IELTS Indicator, LanguageCert, TEF Express, TOEFL iBT Special Home Edition, and Versant. Considering at-home testing more broadly, we discuss key considerations for selecting an at-home test. We close with speculation on how at-home tests may shape language testing going forward: Beyond adapting to the current pandemic, at-home testing might address longstanding issues in access to language testing services and the representation of real-world communication practices in language tests.

30 December 2015 in Applied Linguistics
Verified
Journal Article
Open access
PDF file
BKBenjamin Kremmel
Benjamin Kremmel
TBTineke Brunfaut
Tineke Brunfaut
JAJ. Charles Alderson
J. Charles Alderson
Foreign language (FL) knowledge has been shown to contribute significantly to FL reading performance. Studies have contrasted the contribution of FL vocabulary and syntactic knowledge, following a dichotomous view of these components, producing mixed results. Despite the increasingly recognized formulaic nature of language, the contribution made by phraseological knowledge to reading ability has not been investigated systematically. This study examines the impact of a broader construct definition of linguistic knowledge—which includes a phraseological component—in explaining variance in reading performances. Test scores of 418 learners of English as a foreign language (EFL) were modeled in a structural equation model, showing that a phraseological knowledge measure outperformed traditional syntactic and vocabulary measures in predicting reading comprehension variance. Additional insights into the role of phraseological knowledge were gained through verbal protocol analysis of 15 EFL learners answering reading comprehension items that targeted the understanding of phrasal expressions within written context. The findings hint at an underestimated, but critical, role of phraseological knowledge in FL reading, and are relevant to both the assessment and the teaching of EFL ability.

2 July 2013 in Language Testing
Verified
Journal Article
Closed access
JAJ. Charles Alderson
J. Charles Alderson
BKBenjamin Kremmel
Benjamin Kremmel
“Vocabulary and structural knowledge” (Grabe, 1991, p. 379) appears to be a key component of reading ability. However, is this component to be taken as a unitary one or is structural knowledge a separate factor that can therefore also be tested in isolation in, say, a test of syntax? If syntax can be singled out (e.g. in order to investigate its contribution to reading ability), this test of syntactic knowledge would require validation. The usefulness and reliability of using expert judgments as a means of analysing the content or difficulty of test items in language assessment has been questioned for more than two decades. Still, groups of expert judges are often called upon as they are perceived to be the only or at least a very convenient way of establishing key features of items. Such judgments, however, are particularly opaque and thus problematic when judges are required to make categorizations where categories are only vaguely defined or are ontologically questionable in themselves. This is, for example, the case when judges are asked to classify the content of test items based on a distinction between lexis and syntax, a dichotomy corpus linguistics has suggested cannot be maintained. The present paper scrutinizes a study by Shiotsu (2010) that employed expert judgments, on the basis of which claims were made about the relative significance of the components ‘syntactic knowledge’ and ‘vocabulary knowledge’ in reading in a second language. By both replicating and partially replicating Shiotsu’s (2010) content analysis study, the paper problematizes not only the issue of the use of expert judgments, but, more importantly, their usefulness in distinguishing between construct components that might, in fact, be difficult to distinguish anyway. This is particularly important for an understanding and diagnosis of learners’ strengths and weaknesses in reading in a second language.

24 August 2021 in TESOL Journal
Verified
Journal Article
Open access
PDF file
NSNorbert Schmitt
Norbert Schmitt
KDKaren Dunn
Karen Dunn
BOBarry O’sUllivan
Barry O’sUllivan
LALaurence Anthony
Laurence Anthony
BKBenjamin Kremmel
Benjamin Kremmel
Click on the article title to read more.