Serious games play a significant role in the teaching and learning process by focusing on educational objectives rather than purely on entertainment. By addressing specific educational needs, these games provide targeted learning experiences. The integration of emotion recognition technology into serious games can further enhance teaching and learning by identifying areas where students may need additional support, The integration of emotion recognition into a serious game facilitates the learning of mathematics by allowing the identification of emotional impact on learning and the creation of a tailored learning experience for the student. This study proposes a challenge-based and task-based math serious game that integrates facial emotion recognition named TasksZE. TasksZE introduces a novel approach by adjusting gameplay based on detected emotions, which includes real-time emotion analysis and the cross-validation of emotions. We conducted a usability evaluation of the game using the System Usability Scale (SUS) as a reference, and the results indicate that the students feel that TasksZE is easy to use, the functions are well integrated, and most people can learn to use it very quickly. The students answered that they would use this system frequently since they felt motivated by game attributes, rewards, and level progression. These elements contributed to a more engaging and effective learning experience.