We compared 236 geoscience instructors’ histories of professional development (PD) participation with classroom observations using the Reformed Teaching Observation Protocol (RTOP) that describe undergraduate classes as Student-Centered (score ≥ 50), Transitional (score 31–49) or Teacher-Centered (score ≤ 30). Instructors who attended PD (n = 111) have higher average RTOP scores (44.5 vs. 34.2) and are more frequently observed teaching Student-Centered classes (33% vs. 13%) than instructors with no PD (p < 0.001). Instructors who attended PD that is topically-aligned with content taught during the classroom observation are likely to have RTOP scores that are higher by 13.5 points (p < 0.0001), and are 5.6 times more likely to teach a Student-Centered class than instructors without topically-aligned PD. Comparable odds of teaching Student-Centered classes (5.8x) occur for instructors who attended two topical PD events but were observed teaching a different topic. Models suggest that instructors with at least 24 h of PD are significantly more likely to teach a Student-Centered class than instructors with fewer hours. Our results highlight the effectiveness of discipline-specific PD in impacting teaching practices, and the importance of attending more than one such PD event to aid transfer of learning.
A goal of undergraduate geoscience programs is to prepare students for the geoscience workforce. In order to understand the extent to which programs succeed at this goal, we identified desired workforce skills, compiled undergraduate geoscience program requirements, and mapped workforce skills onto faculty responses to the 2016 National Geoscience Faculty Survey categorized by respondent-provided course names for majors-level courses (n = 1037). We used a Bayesian statistical model to determine the posterior probability of students practicing desired workforce skills in undergraduate geoscience programs, using program profiles as the model prior, conditioned by the likelihood of a student practicing workforce skills in each course category based upon survey responses. Probabilistic Monte Carlo simulations of students in each example program were used to calculate the likely number of courses in which a student practiced each skill. Activities to help students develop skills related to geologic reasoning, working as part of a team, quantitative skills (algebra), applying skills in new scenarios, evaluation of scientific literature, temporal thinking, spatial thinking, written communication, and managing uncertainty are frequently encountered by students; on average, students encounter these activities at least seven times during an undergraduate program. In contrast, students encounter activities to help them develop an understanding of societal relevance and systems thinking, on average, fewer than three times during an undergraduate program. Our results provide a snapshot, via instructors’ reporting, of the state of workforce skill development across undergraduate degree programs, and both the approach and data set can be built upon for future studies.
28 October 2020 in International Journal of STEM Education
Background: The Classroom Observation Project employs direct observations of geoscience teaching across the USA using the Reformed Teaching Observation Protocol (RTOP) to quantify the use of reformed teaching practices. We report on 345 RTOP observations used to evaluate the extent of teaching reform when curricular materials developed as part of the InTeGrate Project (ITG) were used. The InTeGrate Project has published 40 modules of curricular materials that teach geoscience in the context of societal issues and support instructors through guided use of student-centered instructional practices. All ITG materials were developed by teams of instructors, follow a consistent structure, and were evaluated against a project rubric.Results: RTOP scores for classes observed when ITG materials were used (ITG;n= 50,M= 54.0) are significantly higher than RTOP scores for classes observed when ITG materials were not used (non-ITG;n= 295;M= 39.8;p< .0001). ITG observations all have RTOP scores in the student-centered (≥ 50) or transitional (31–49) instructional categories, and none in the teacher-centered instructional category (≤ 30), demonstrating that ITG materials support more student-centered teaching in class sessions where they are used. In 33 paired observations of the same instructor teaching with and without ITG materials, mean RTOP scores when teaching with ITG are greater than mean RTOP scores when teaching without ITG (M= 54 andM= 47.1, respectively).Conclusions: RTOP observations reveal that more student-centered instructional practices occur in class sessions in which ITG materials are used. There is a small range of RTOP scores when individual ITG activities are used by multiple instructors, suggesting that using ITG materials results in a consistent quality of instruction. The complete absence of teacher-centered instruction when using ITG materials means the materials are a useful resource for practicing reformed teaching methods. The model of the ITG Project in the creation and broad dissemination of ready-made curricula for use in large numbers of classrooms can be replicated to transform teaching and learning in other disciplinary communities.
15 January 2022 in International Journal of STEM Education
Background: Efforts to achieve improved student outcomes in STEM are critically reliant on the success of reform efforts associated with teaching and learning. Reform efforts include the transformation of course-based practices, community values, and the institutional policies and structures associated with teaching and learning in higher education. Enacting change is a complex process that can be guided by change theories that describe how and why a desired change takes place. We analyzed the utility of a theory-based change model applied in a higher education setting. Our results provide guidance for change efforts at other institutions.Results: Use of the CACAO model to guide the transformation of STEM instruction at a large public university resulted in changes to faculty teaching practices and department culture consistent with the vision defined for the project. Such changes varied across STEM departments in accordance with the emergent nature of project activities at the department level. Our application of the CACAO model demonstrates the importance of (1) creating a vision statement (statement of desired change or end-state); (2) attending to different levels of the organization (e.g., individuals, departments, and colleges); (3) working with change agents who are situated to be effective at different organizational levels; and (4) employing strategies to meet the needs and interests of faculty at different stages of adoption with respect to the desired change.Conclusion: Our work, which demonstrates the utility of the CACAO model for change and captures its key elements in a matrix, provides a potential foundation for others considering how to frame and study change efforts. It reinforces the value of using change theories to inform change efforts and creates a structure that others can build on and modify, either by applying our CACAO matrix in their own setting or by using the matrix to identify elements that connect to other change theories. We contribute to the growing body of literature which seeks to understand how change theories can be useful and generalizable beyond a single project.
19 December 2019 in Journal of Geoscience Education
Quantitative literacy is a foundational component of success in STEM disciplines and in life. Quantitative concepts and data-rich activities in undergraduate geoscience courses can strengthen geoscience majors’ understanding of geologic phenomena and prepare them for future careers and graduate school, and provide real-world context to apply quantitative thinking for non-STEM students. We use self-reported teaching practices from the 2016 National Geoscience Faculty Survey to document the extent to which undergraduate geoscience instructors emphasize quantitative skills (algebra, statistics, and calculus) and data analysis skills in introductory (n = 1096) and majors (n = 1066) courses. Respondents who spent more than 20% of class time on student activities, questions, and discussions, taught small classes, or engaged more with the geoscience community through research or improving teaching incorporated statistical analyses and data analyses more frequently in their courses. Respondents from baccalaureate institutions reported use of a wider variety of data analysis skills in all courses compared with respondents from other types of institutions. Additionally, respondents who reported using more data analysis skills in their courses also used a broader array of strategies to prepare students for the geoscience workforce. These correlations suggest that targeted professional development could increase instructors’ use of quantitative and data analysis skills to meet the needs of their students in context.