1 February 2023 in Psychological Test Adaptation and Development
2 October 2020 in Frontiers in Psychology
In this paper, we compare the performance of 18 indicators of cheating on e-exams in higher education. Basis of the study was a field experiment. The experimental setting was a computer assisted mock exam in an introductory course on psychology conducted at a university. The experimental manipulation consisted in inducing two forms of cheating (pre-knowledge, test collusion) in a subgroup of the examinees. As indicators of cheating, we consider well-established person-fit indices (e.g., the U3 statistic), but also several new ones based on process data (e.g., response times). The indicators were evaluated with respect to their capability to separate the subgroup of the cheaters from the remaining examinees. We additionally employed a classification tree for detecting the induced cheating behavior. With this proceeding, we aimed at investigating the detectability of cheating in the day-to-day educational setting where conditions are suboptimal (e.g., tests with low psychometric quality are used). The indicators based on the number of response revisions and the response times were capable to indicate the examinees who cheated. The classification tree achieved an accuracy of 0.95 (sensitivity: 0.42/specificity: 0.99). In the study, the number of revisions was the most important predictor of cheating. We additionally explored the performance of the indicators to predict the specific form of cheating. The specific form was identified with an accuracy of 0.93.
2 February 2013 in Psychometrika
Findings suggest that in psychological tests not only the responses but also the times needed to give the responses are related to characteristics of the test taker. This observation has stimulated the development of latent trait models for the joint distribution of the responses and the response times. Such models are motivated by the hope to improve the estimation of the latent traits by additionally considering response time. In this article, the potential relevance of the response times for psychological assessment is explored for the model of van der Linden (Psychometrika 72:287–308, 2007) that seems to have become the standard approach to response time modeling in educational testing. It can be shown that the consideration of response times increases the information of the test. However, one also can prove that the contribution of the response times to the test information is bounded and has a simple limit.
1 September 2015 in Psychometrika
Latent trait models for responses and response times in tests are often pure statistical models without a close connection to features of the assumed response process. In the present paper, a new model is presented that is more closely related to assumptions about the response process. The model is based on two increasing stochastic processes. Each stochastic process represents the accumulation of knowledge with respect to one of two response options, the correct and incorrect response. Both accumulators compete and the accumulator that first exceeds a critical level determines the response. General assumptions about the accumulators result in a race between two response times that follow a bivariate Birnbaum Saunders distribution. The model can be calibrated with marginal maximum likelihood estimation. Feasibility of the estimation approach is demonstrated in a simulation study. Additionally, a test of model fit is proposed. Finally, the model will be used for the analysis of an empirical data set.
17 March 2020 in Educational and Psychological Measurement
The hierarchical model of van der Linden is the most popular model for responses and response times in tests. It is composed of two separate submodels—one for the responses and one for the response times—that are joined at a higher level. The submodel for the response times is based on the lognormal distribution. The lognormal distribution is a skew distribution with a support from zero to infinity. Such a support is unrealistic as the solution process demands a minimal processing time that sets a response time threshold. Ignoring this response time threshold misspecifies the model and threatens the validity of model-based inferences. In this article, the response time model of van der Linden is replaced by a model that is based on the three-parameter lognormal distribution. The three-parameter lognormal distribution extends the lognormal distribution by an additional location parameter that bounds the support away from zero. Two different approaches to model fitting are proposed and evaluated with regard to parameter recovery in a simulation study. The extended model is applied to two data sets. In both data sets, the extension improves the fit of the hierarchical model.
2 September 2013 in British Journal of Mathematical and Statistical Psychology