Self-report surveys are extensively utilized in educational research to understand students’ perceptions and experiences. However, younger children, particularly those in elementary school, may exhibit a tendency to provide socially desirable responses, potentially compromising the data quality. This study examined the prevalence and impact of socially desirable responses in self-report surveys administered to elementary school students. A total of 1,024 students from grades 4 and 5 in five elementary schools participated in the study. Socially desirable responses were measured using detection items embedded within questionnaires. The findings indicate that (a) more than 20% of elementary school students demonstrated socially desirable responses; (b) female students and those with higher academic achievement were more likely to provide socially desirable responses; (c) socially desirable responses skewed the sample distribution by inflating mean scores and reducing standard deviations; and (d) while internal correlations within scales remained relatively stable, external validity, as reflected in correlations between self-reports and academic performance metrics, was significantly affected after adjusting for socially desirable responses. These results underscore the importance of addressing socially desirable responses when interpreting self-report data from young students. The study concludes with practical recommendations for improving the validity of self-report surveys in educational research.
The popularity of self-report surveys brings to light the issue of insufficient effort responding (IER), where participants lack motivation to respond thoughtfully, thereby compromising data quality. A prevalent form of IER, known as straightlining, occurs when participants consistently select the same response for multiple items, leading to potentially biased outcomes. Typically, researchers apply a uniform threshold criterion across all response options (e.g., nine consecutive identical answers in any option are considered problematic). Considering the diverse or near-normal distribution patterns in questionnaire responses, employing varied thresholds for different options may be more appropriate. This study utilized both simulated data and real, large-scale international survey data to examine the likelihood of straightlining as IER across various options. The findings revealed that legitimate straightlining (non-IER) was more common at the extremes than in the middle options. Moreover, the study highlights the significant risks of incorrectly categorizing non-IER straightlining as IER without corroborating evidence. In conclusion, our results question the conventional approaches to eliminating straightlining, including (a) the application of a uniform cutoff value across different options or adopting stricter criteria for middle options and (b) the direct removal of straightlining as IER without additional verification.
Detecting participants not exerting sufficient effort (e.g., answering randomly; termed 'insufficient effort responding' or IER) in self -report surveys is crucial but intricate. Challenges of appropriately removing IER resemble a surgeon excising necrotic tissue without being inadequate, excessive, or incorrect. Current cutoff criteria for stopping the elimination of suspected IER responses are often arbitrary. This study proposes an external criterion strategy to identify the optimal cutoff values for various IER detection methods. We investigated the change in correlations between the IER-containing scale (e.g., motivation) and the external criterion (e. g., academic performance, socioeconomic status index, or another questionnaire scale) utilizing simulated and authentic international survey data. The findings revealed that the stopping rule and consequently, the optimal cutoff values for IER detection methods, can be accurately identified by locating the inflection point in the correlation plot with the external criterion. Practical strategies are recommended for applied researchers.
Computational thinking (CT) currently has been mainly explored using programming robots and conducted in K12 education. In early childhood education, arts have a significant place in children’s learning, expression and cognitive development. Specifically, creating pop-up storybooks is a child-friendly activity. Our study aimed to explore the combination of CT and art, and to develop talented children’s CT abilities through a multimodal literacies educational approach, which referred to using different skills (i.e., writing, drawing, making, and storytelling) in the art activity. A total of 12 talented children were selected to participate in a pop-up storybook production workshop using a convenience sampling method. We adopted an observation method to capture talented children’s CT behaviors, generating a total of 180 min of activity videos and collecting fieldnotes and the children’s worksheets and artworks for the data analysis. Based on a content analysis, we found that talented children enhanced their CT development in multiple modes and practiced seven CT skills. In conclusion, our study emphasizes the importance of art in children’s education and provides new insights for subsequent research on children’s CT education.
24 July 2025 in Large-scale Assessments in Education
Although self-report surveys are widely used for data collection, data quality can vary across populations because certain groups are more likely to engage in insufficient effort responding (IER). Our study examined how different levels of the educational system—student groups, schools, and cultural contexts—affect data quality due to IER, using a three-level Poisson regression analysis of the PISA 2018 dataset. We observed IER prevalence ranging from 14 to 74% in PISA questionnaire, depending on the indicators used. Notably, students with low academic performance were more likely to engage in IER. Additionally, students from low academic performance or high-SES schools exhibited higher IER tendencies, while IER showed minimal variation across cultural contexts. These findings emphasized that IER introduced systematic biases into survey data, undermining the fairness of group comparisons and confounding results with participants’ characteristics. Our study advocated for a careful approach to addressing IER to enhance the validity of self-report measurements.