This article investigates the application of a deep learning model for evaluating the conformity of model images to types of UML diagrams to be used in self-training and educational settings. Our approach leans on a feature-based dataset that captures a broad range of modeling elements from class, state machine, and sequence diagrams, enhancing the ability to recognize a larger variety of categories selected for this research. The neural network trained with these features representing parts of the UML concrete syntax demonstrates 90% in classification accuracy on average, in respect to our previous research on UML diagrams classification without using a feature-based dataset. This study concludes that a feature-based approach, combined with advanced neural network architectures, can improve the classification of such images, especially in edge cases where diagrams contain similar graphical details but the whole does not represent a UML diagram. For the given research, we obtained a 0.87 F1 score.
This article investigates the use of neural network-based UML diagram classification to support automated assessment in software engineering education; this may be useful to both teachers and students. For this purpose, our software automatically classifies UML models that conform to the standard Unified Modeling Language (UML), using neural networks trained on an independent dataset composed of public UML repositories and collected student models; thus, starting from models provided by students as images, it distinguishes between various types of UML diagrams, and it also detects whether they do not conform to the standard language. The case study includes a detailed analysis performed for three cohorts totaling 237 students to identify correct assessments, recurring errors, and the impact of diagram quality, missing elements, duplicates, and non-standard notations. To perform an evaluation under imbalanced data conditions, macro average metrics were applied, including precision, recall, specificity, and F1 score. Across the complete evaluation dataset of 881 student submissions, the assessment software achieved a macro average F1 score of 0.96 across the four evaluable ground-truth classes (class, sequence, state, and other UML); the non-UML class was excluded from this macro average because the classroom dataset contained no ground-truth non-UML submissions. These findings demonstrate the feasibility and the limitations of AI-based assessment of students’ UML modeling practical work and highlight opportunities for future integration into student self-assessment and intelligent tutoring systems.