7 November 2012 in Perspectives on Psychological Science
Recent controversies in psychology have spurred conversations about the nature and quality of psychological research. One topic receiving substantial attention is the role of replication in psychological science. Using the complete publication history of the 100 psychology journals with the highest 5-year impact factors, the current article provides an overview of replications in psychological research since 1900. This investigation revealed that roughly 1.6% of all psychology publications used the term replication in text. A more thorough analysis of 500 randomly selected articles revealed that only 68% of articles using the term replication were actual replications, resulting in an overall replication rate of 1.07%. Contrary to previous findings in other fields, this study found that the majority of replications in psychology journals reported similar findings to their original studies (i.e., they were successful replications). However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. Moreover, despite numerous systemic biases, the rate at which replications are being published has increased in recent decades.
31 March 2021 in Educational Psychologist
Replication is a key activity in scientific endeavors. Yet explicit replications are rare in many fields, including education and psychology. In this article, we discuss the relevance and value of replication in educational psychology and analyze challenges regarding the role replications can and should play in research. These challenges include philosophical, methodological, professional, and utility concerns about replication in education and the social sciences more broadly. Finally, we discuss strategies that may address these concerns in educational psychology research.
1 April 2019 in AERA Open
Educators have sought to understand and address the disproportional representation of students from certain student subgroups in gifted education. Most gifted identification decisions are made with national comparisons where students must score above a certain percentage of test takers. However, this approach is not always consistent with the overall goal of gifted education. Scholars have long argued for the use of local normative criteria to increase the diversity of students identified for gifted services, and although some districts across the country have applied such recommendations, little research has been carried out. In this study, we use a large data set to assess the extent to which identifying gifted students with either school-level norms or a combination of national and school-level norms would improve gifted education representation rates for students who are from African American and Latinx families. A preprint of this registered report and this project’s preregistration documentation are available at https://osf.io/z2egy/ .
25 May 2016 in Psychological Science
The educational, occupational, and creative accomplishments of the profoundly gifted participants (IQs ⩾ 160) in the Study of Mathematically Precocious Youth (SMPY) are astounding, but are they representative of equally able 12-year-olds? Duke University’s Talent Identification Program (TIP) identified 259 young adolescents who were equally gifted. By age 40, their life accomplishments also were extraordinary: Thirty-seven percent had earned doctorates, 7.5% had achieved academic tenure (4.3% at research-intensive universities), and 9% held patents; many were high-level leaders in major organizations. As was the case for the SMPY sample before them, differential ability strengths predicted their contrasting and eventual developmental trajectories—even though essentially all participants possessed both mathematical and verbal reasoning abilities far superior to those of typical Ph.D. recipients. Individuals, even profoundly gifted ones, primarily do what they are best at. Differences in ability patterns, like differences in interests, guide development along different paths, but ability level, coupled with commitment, determines whether and the extent to which noteworthy accomplishments are reached if opportunity presents itself.
2 May 2016 in Remedial and Special Education
Increased calls for rigor in special education have often revolved around the use of experimental research design. However, the replicability of research results is also a central tenet to the scientific research process. To assess the prevalence, success rate, and authorship history of replications in special education, we investigated the complete publication history of every replication published in the 36 journals categorized by ISI Web of Knowledge Journal Citation Report as special education. We found that 0.5% of all articles reported seeking to replicate a previously published finding. More than 80% of these replications reported successfully replicating previous findings. However, replications where there was at least one author overlapping with the original article (which happens about two thirds of the time) were statistically significantly more likely to find successful results.
10 March 2021 in Educational Researcher
Concerns about the conduct of research are pervasive in many fields, including education. In this preregistered study, we replicated and extended previous studies from other fields by asking education researchers about 10 questionable research practices and five open research practices. We asked them to estimate the prevalence of the practices in the field, to self-report their own use of such practices, and to estimate the appropriateness of these behaviors in education research. We made predictions under four umbrella categories: comparison to psychology, geographic location, career stage, and quantitative orientation. Broadly, our results suggest that both questionable and open research practices are used by many education researchers. This baseline information will be useful as education researchers seek to understand existing social norms and grapple with whether and how to improve research practices.