Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Online questionnaires are widely used for large-scale mental health screening. However, careless responses from participants can compromise data validity, ultimately undermining the reliability of screening outcomes. Prior studies have primarily focused on the effects of individual and environmental factors on careless responding, the heterogenous effect role of questionnaire type remains underexplored. Furthermore, few studies have examined careless responding within mental health screening contexts. Objective: This study aims to investigate the individual factors influencing careless responses in online mental health screening and assess how the effect of these factors varies across different types of psychological screening instruments. Methods: This study analyzes data from 24,367 participants at Xinxiang Medical University across four psychological questionnaires (PHQ-9, PSS, ISI, GAD-7). Careless responding is defined as a continuous dependent variable, measured by the proportion of items completed in less than two seconds per item — a widely accepted criterion for identifying insufficient response time. We employ multiple linear regression model to examine the effect of individual factors (gender, age, education, smoking, and drinking) on careless responding while assessing variations across questionnaire types. Additionally, response time distributions are visualized to identify typical patterns between careless and careful responders across four psychological questionnaires. Results: Females consistently demonstrate lower levels of careless responding than males, when completing the PHQ-9 (β=-0.172, 95% CI -0.104 to -0.089, P<.001), PSS (β=-0.234, 95% CI -0.162 to -0.144, P<.001), ISI (β=-0.207, 95% CI -0.13 to -0.114, P<.001), and GAD-7 (β=-0.177, 95% CI -0.108 to -0.093, P<.001). While older participants demonstrate lower levels of careless responding when completing the PHQ-9 (β=-0.036, 95% CI -0.007 to -0.003], P<.001), ISI (β=-0.036, 95% CI -0.007 to -0.003, P<.001) and GAD-7 (β=-0.053, 95% CI -0.009 to -0.005], P<.001), their age is unrelated to careless responding on the PSS. Interestingly, compared with participants with a associate-level education, high education background (bachelor, master, doctor) consistently demonstrates high-level of careless responding, especially for master’s degree holders (PHQ-9: β=0.098, 95% CI 0.136 to 0.188], P<.001; GAD-7: β=0.091, 95% CI 0.125 to 0.178, P<.001). Smokers exhibit varied patterns, with current smokers demonstrate lower levels of careless responding the PHQ-9 (β=-0.022, 95% CI -0.064 to -0.016, P=.001) and GAD-7 (β=-0.014, 95% CI -0.051 to -0.002, P=.03) but occasional smokers demonstrate higher levels of careless responding the PSS (β=0.019, 95% CI 0.010 to 0.050, P=.003) than non-smokers. Drinkers consistently demonstrate lower levels of careless responding than non-drinkers, with the strongest effect for drinks occasionally on the PHQ-9 (β=-0.163, 95% CI -0.103 to -0.087, P<.001). Analysis of response times revealed that participants tended to spend less time on surveys on depression and anxiety, and careless responding in surveys on stress and insomnia was characterized by skipping questions. Conclusions: The effect of individual factors varies across questionnaire types. These findings offer valuable insights for questionnaire designers and administrators, highlighting the need for targeted interventions tailored to user group characteristics and the unique features of different questionnaire types to mitigate careless responding.