Instructors often incorporate self- and peer evaluations when they use teamwork in their classes, which is common in management education. However, the process is often time consuming and frequently does not match well with guidance provided by the literature. We describe the development of a web-based instrument that efficiently collects and analyzes self- and peer-evaluation data. The instrument uses a behaviorally anchored rating scale to measure team-member contributions in five areas based on the team effectiveness literature. Three studies provide evidence for the validity of the new instrument. Implications for management education and areas for future research are discussed.
This article describes the development of the Comprehensive Assessment of Team Member Effectiveness. The authors used the teamwork literature to create potential items, which they tested using two surveys of college students (Ns = 2,777 and 1,157). The authors used exploratory factor analysis and confirmatory factor analysis to help them select items for the final instrument. The full instrument has 87 items that measure 29 types of team member contributions with 3 items each. These fall into five categories (contributing to the team's work; interacting with teammates; keeping the team on track; expecting quality; and having relevant knowledge, skills, and abilities). A short version of the instrument has 33 items. Potential uses for the instrument and suggestions for future research are discussed.
2 January 2010 in Journal of Engineering Education
Management researchers often use consensus-based composition models to examine the antecedents and effects of higher-level constructs. Typically, researchers present three indices, rwg, ICC(1), and ICC(2), to demonstrate agreement and consistency among lower-level units when justifying aggregation. Nevertheless, researchers debate what values for these indices are sufficient. This study examines the distributional characteristics of ICCs and rwg values from three sources: the multilevel literature, a large multinational sample of student teams, and a large sample of randomly generated “pseudo teams.” Our results support existing cutoff criteria for ICCs but suggest that generally accepted values for rwg may, under certain circumstances, reflect pseudo-agreement (i.e., agreement observed among two raters not attributable to the same target). Thus, when there is minimal between-group variance (i.e., low ICCs), it is difficult to determine whether high rwg values reflect agreement or pseudo-agreement. Based on these findings, we provide recommendations to help researchers interpret aggregation indices.
Colleges of business must meet assurance of learning requirements to gain or maintain AACSB accreditation under the new standards adopted April 8, 2013. Team skills are among the most important skills desired by recruiters, yet employers and scholars perceive that team skills are frequently deficient in college graduates. This article describes how a set of free web-based tools available at www.CATME.org allows colleges to collect data to determine whether their students are developing good team skills and demonstrate achievement for accreditation reviews for learning goals related to team skills. The tools, which were developed to support teamwork in college classes, allow instructors to easily collect, interpret, and share data about students and teams. We show how colleges that use the CATME system in all courses requiring teamwork could analyze the data to demonstrate that their programs develop students’ team skills. The appendix provides 13 examples of learning goals colleges could use, along with a method to demonstrate achievement for each, and a sample measurable program target. These examples relate to team skills such as working with teammates of diverse backgrounds, contributing effectively to teams, displaying good interpersonal skills, being able to evaluate teamwork, and recognizing team processes and outcomes.