1 March 2008 in American Journal of Evaluation
This study examines response rates for mixed-mode survey implementation involving mail and e-mail/Web components. Using Dillman's Tailored Design Method, 1,500 participants were sent a survey either (a) via mail with a follow-up contact via e-mail that directed them to a Web-based questionnaire or (b) via e-mail that directed them to a Web-based questionnaire with a follow-up contact via mail. Results indicate that these mixed-mode procedures produce moderately high response rates. However, the mail survey tended to be more effective than the e-mail/Web survey, when serving either as the initial contact or as the follow-up contact. These results suggest that survey implementation involving mail followed by e-mail/Web, or even mail-only approaches, may result in larger samples than implementation involving e-mail/Web followed by mail.
2 December 2009 in Journal of Educational Measurement
1 October 1998 in Written Communication
This article examines the behavioral differences of essay scorers who demonstrate different levels of proficiency for a psychometric scoring task. The authors compare three proficiency groups to identify differences in (a) essay features they consider, (b) their understanding of the scoring rubric, and (c) their decision-making procedures. Results indicate scorers with different levels of proficiency do not focus on different essay features when making evaluative decisions but their understandings of the scoring criteria may vary. Proficient scores are more likely to focus on general features of an essay when making evaluative decisions and to adopt values espoused by the scoring rubric than are less proficient scorers. Also, proficient scorers make evaluations by reading the entire essay and then reviewing its content, whereas less proficient scorers may interrupt the reading process to monitor how well the essay satisfies the scoring criteria. Finally, the authors discuss implications for scorer selection and training.
18 September 2012 in Educational Measurement: Issues and Practice
1 January 1997 in Assessing Writing
1 September 2005 in Measurement in Physical Education and Exercise Science
This study extends validity evidence for the Coaching Efficacy Scale (CES; Feltz, Chase, Moritz, & Sullivan, 1999) by providing an evaluation of the psychometric properties of the instrument from previously collected data on high school and college coaches from United States. Data were fitted to a multidimensional item response theory model. Results offered some supporting evidence concerning validity based on the fit of a multidimensional conceptualization of coaching efficacy (i.e., motivation, game strategy, technique, and character building) as compared to a unidimensional conceptualization of coaching efficacy (i.e., total coaching efficacy), the fit of the majority of items to the measurement model, the internal consistency of coaching efficacy estimates, and the precision of total coaching efficacy estimates. However, concerns exist relating to the rating scale structure, the precision of multidimensional coaching efficacy estimates, and misfit of a couple of items to the measurement model. Practic...