According to the recent Nation’s Report Card, 12th-graders failed to produce gains on the 2005 National Assessment of Educational Progress (NAEP) despite earning better grades on average. One possible explanation is that 12th-graders were not motivated taking the NAEP, which is a low-stakes test. We develop three Bayesian IRT mixture models to describe the results from a group of examinees including both nonguessers and partial guessers. The first assumes that the guesser answers questions based on his or her knowledge up to a certain test item, and guesses thereafter. The second model assumes that the guesser answers relatively easy questions based on his or her knowledge and guesses randomly on the remaining items. The third is constructed to describe more general low-motivation behavior. It assumes that the guesser gives less and less effort as he or she proceeds through the test. The models can provide not only consistent estimates of IRT parameters but also estimates of each examinee’s nonguesser/guesser status and degree of guessing behavior. We show results of a simulation study comparing the performance of the three guessing models to the 2PL-IRT model. Finally, an analysis of real data from a low-stakes test administered to university students is presented.
Many researchers use the double filtering procedure with fold change and t test to identify differentially expressed genes, in the hope that the double filtering will provide extra confidence in the results. Due to its simplicity, the double filtering procedure has been popular with applied researchers despite the development of more sophisticated methods.
1 December 2009 in Journal of Statistical Planning and Inference
Judge performance is a critical component of a wine competition's success. A number of studies have shown that wine judges may differ considerably in their opinions. In this paper, we have conducted an in-depth examination of wine judge performance at a U.S. wine competition. Three characteristics of judge's performance are examined: bias, discrimination ability, and variation. Based on the analysis, we can identify the judges who had discrepant scoring patterns and can gain insight into which of the three characteristics cause particular judges to disagree. The evaluation of wine judge performance through these three characteristics may provide useful information for training them to have consistent performance and in assisting competition coordinators in judge selection. (JEL Classification: C1, D8, Q13)
The purpose of this paper is to provide a tutorial of data analysis methods for answering questions that arise in analyzing data from wine-tasting events: (i) measuring agreement of two judges and its extension to m judges; (ii) making comparisons of judges across years; (iii) comparing two wines; (iv) designing tasting procedures to reduce burden of multiple tastings; (v) ranking of judges; and (vi) assessing causes of disagreement. In each case we describe one or more analyses and make recommendations on the conditions of use for each. (JEL Classifications: C10, C12, C13, C59, C90)