Improving student performance and fostering higher-order abilities in physics experiment courses has long been difficult, since such courses require both conceptual mastery and hands-on problem-solving. This study has compared three instructional approaches—traditional blended learning (control group, CG), flipped classroom-based blended learning (FB), and problem-based blended learning (PB)—to evaluate their impact on the students’ academic performance, critical thinking, and self-efficacy. A quasi-experimental pretest–posttest design was implemented with second-year undergraduates. The data was collected using standardised academic assessments, validated psychological scales, and behavioural learning records. The findings indicated that FB and PB produced better results than CG. FB resulted in more significant improvements in critical thinking (Cohen’s d = 0.855), while PB was particularly helpful in boosting self-efficacy (Cohen’s d = 0.842). The multiple regression results indicated that the behavioural indicators, including activity performance and experiment reports, strongly predict academic achievement in FB and PB. At the same time, the initial scores played a larger role in CG. Beyond statistical significance, this study points to clear pedagogical implications. Instructors can make blended courses more responsive and equitable by embedding behavioural learning analytics into curriculum design. The results call for a transformation in higher education, advancing dynamic, learner-centred, and personalised practices that foster innovation and equity in STEM education.