Andreea Alexandra Anghel

Ms. Andreea Alexandra Anghel

orcid

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
31 July 2025 in Information
Verified
Journal Article
Open access
PDF file
Catalin AnghelCatalin Anghel
Catalin Anghel
AAAndreea Alexandra Anghel
Andreea Alexandra Anghel
EPEmilia Pecheanu
Emilia Pecheanu
ACAdina Cocu
Adina Cocu
AIAdrian Istrate
Adrian Istrate
CAConstantin Adrian Andrei
Constantin Adrian Andrei
The evaluation of large language models (LLMs) increasingly relies on other LLMs acting as automated judges. While this approach offers scalability and efficiency, it raises serious concerns regarding evaluator reliability, positional bias, and ranking stability. This paper presents a scalable framework for diagnosing positional bias and instability in LLM-based evaluation by using controlled pairwise comparisons judged by multiple independent language models. The system supports mirrored comparisons with reversed response order, prompt injection, and surface-level perturbations (e.g., paraphrasing, lexical noise), enabling fine-grained analysis of evaluator consistency and verdict robustness. Over 3600 pairwise comparisons were conducted across five instruction-tuned open-weight models using ten open-ended prompts. The top-performing model (gemma:7b-instruct) achieved a 66.5% win rate. Evaluator agreement was uniformly high, with 100% consistency across judges, yet 48.4% of verdicts reversed under mirrored response order, indicating strong positional bias. Kendall’s Tau analysis further showed that local model rankings varied substantially across prompts, suggesting that semantic context influences evaluator judgment. All evaluation traces were stored in a graph database (Neo4j), enabling structured querying and longitudinal analysis. The proposed framework provides not only a diagnostic lens for benchmarking models but also a blueprint for fairer and more interpretable LLM-based evaluation. These findings underscore the need for structure-aware, perturbation-resilient evaluation pipelines when benchmarking LLMs. The proposed framework offers a reproducible path for diagnosing evaluator bias and ranking instability in open-ended language tasks. Future work will apply this methodology to educational assessment tasks, using rubric-based scoring and graph-based traceability to evaluate student responses in technical domains.

11 October 2025 in Computers
Verified
Journal Article
Open access
PDF file
Catalin AnghelCatalin Anghel
Catalin Anghel
MCMarian Viorel Crăciun
Marian Viorel Crăciun
EPEmilia Pecheanu
Emilia Pecheanu
ACAdina Cocu
Adina Cocu
AAAndreea Alexandra Anghel
Andreea Alexandra Anghel
Serban DragosloveanuSerban Dragosloveanu
Serban Dragosloveanu
Background and objectives: Large language models (LLMs) show promise in automating open-ended evaluation tasks, yet their reliability in rubric-based assessment remains uncertain. Variability in scoring, feedback, and rubric adherence raises concerns about transparency and pedagogical validity in educational contexts. This study introduces CourseEvalAI, a framework designed to enhance consistency and fidelity in rubric-guided evaluation by fine-tuning a general-purpose LLM with authentic university-level instructional content. Methods: The framework employs supervised fine-tuning with Low-Rank Adaptation (LoRA) on rubric-annotated answers and explanations drawn from undergraduate computer science exams. Responses generated by both the base and fine-tuned models were independently evaluated by two human raters and two LLM judges, applying dual-layer rubrics for answers (technical or argumentative) and explanations. Inter-rater reliability was reported as intraclass correlation coefficient (ICC(2,1)), Krippendorff’s α, and quadratic-weighted Cohen’s κ (QWK), and statistical analyses included Welch’s t tests with Holm–Bonferroni correction, Hedges’ g with bootstrap confidence intervals, and Levene’s tests. All responses, scores, feedback, and metadata were stored in a Neo4j graph database for structured exploration. Results: The fine-tuned model consistently outperformed the base version across all rubric dimensions, achieving higher scores for both answers and explanations. After multiple-testing correction, only the Generative Pre-trained Transformer (GPT-4)—judged Technical Answer contrast remains statistically significant; other contrasts show positive trends without passing the adjusted threshold, and no additional significance is claimed for explanation-level results. Variance in scoring decreased, inter-model agreement increased, and evaluator feedback for fine-tuned outputs contained fewer vague or critical remarks, indicating stronger rubric alignment and greater pedagogical coherence. Inter-rater reliability analyses indicated moderate human–human agreement and weaker alignment of LLM judges to the human mean. Originality: CourseEvalAI integrates rubric-guided fine-tuning, dual-layer evaluation, and graph-based storage into a unified framework. This combination provides a replicable and interpretable methodology that enhances the consistency, transparency, and pedagogical value of LLM-based evaluators in higher education and beyond.

1 August 2025 in Informatics
Verified
Journal Article
Open access
PDF file
Catalin AnghelCatalin Anghel
Catalin Anghel
AAAndreea Alexandra Anghel
Andreea Alexandra Anghel
EPEmilia Pecheanu
Emilia Pecheanu
ISIoan Susnea
Ioan Susnea
ACAdina Cocu
Adina Cocu
AIAdrian Istrate
Adrian Istrate
(1) Background and objectives: Large language models (LLMs) such as GPT, Mistral, and LLaMA exhibit strong capabilities in text generation, yet assessing the quality of their reasoning—particularly in open-ended and argumentative contexts—remains a persistent challenge. This study introduces Dialectical Agent, an internally developed modular framework designed to evaluate reasoning through a structured three-stage process: opinion, counterargument, and synthesis. The framework enables transparent and comparative analysis of how different LLMs handle dialectical reasoning. (2) Methods: Each stage is executed by a single model, and final syntheses are scored via two independent LLM evaluators (LLaMA 3.1 and GPT-4o) based on a rubric with four dimensions: clarity, coherence, originality, and dialecticality. In parallel, a rule-based semantic analyzer detects rhetorical anomalies and ethical values. All outputs and metadata are stored in a Neo4j graph database for structured exploration. (3) Results: The system was applied to four open-weight models (Gemma 7B, Mistral 7B, Dolphin-Mistral, Zephyr 7B) across ten open-ended prompts on ethical, political, and technological topics. The results show consistent stylistic and semantic variation across models, with moderate inter-rater agreement. Semantic diagnostics revealed differences in value expression and rhetorical flaws not captured by rubric scores. (4) Originality: The framework is, to our knowledge, the first to integrate multi-stage reasoning, rubric-based and semantic evaluation, and graph-based storage into a single system. It enables replicable, interpretable, and multidimensional assessment of generative reasoning—supporting researchers, developers, and educators working with LLMs in high-stakes contexts.

29 May 2026 in AI
Verified
Journal Article
Open access
PDF file
Catalin AnghelCatalin Anghel
Catalin Anghel
AAAndreea Alexandra Anghel
Andreea Alexandra Anghel
MCMarian Viorel Crăciun
Marian Viorel Crăciun
ACAdina Cocu
Adina Cocu
DVDiana Elena Vulpe
Diana Elena Vulpe
Romica CerganRomica Cergan
Romica Cergan
Large Language Models (LLMs) are increasingly used for rubric-based assessment, but reliable automated grading requires more than a single prompt-response step. This study presents GradeAgentOps, a verification-first framework for evidence-anchored LLM exam grading that combines strict grading contracts, deterministic verification and canonicalization, bounded semantic repair, optional memory modules, and provenance-aware logging. The framework is evaluated on a university-level dataset of 1000 short open-ended exam responses from 100 students across 10 questions, with annotations from two independent expert human graders. A controlled ablation protocol compares six configurations, including a rubric-only baseline and progressively stronger variants with repair and memory augmentation. Human–human agreement provides the reference context, with overall ICC(2,1) = 0.678 and QWK = 0.678 between the two graders. Using one expert grader as the operational reference, the FULL configuration achieves the strongest model-human agreement (MAE = 1.935, RMSE = 2.500, QWK = 0.652, Within ±2 = 0.667), with the consistency memory configuration with repair (C1) emerging as the closest alternative. The gains are not uniform: improvements are more pronounced on argumentative items than on technical ones, and pairwise comparisons show that consistency memory contributes more strongly than rubric memory alone. The stronger configurations also produce cleaner accepted outputs, with lower rates of evidence-related postprocess issues, although at a moderate operational cost. Overall, within this controlled single-course evaluation setting, the results show that reliable automated grading benefits not only from model capability, but also from pipeline design choices that promote verification, evidential coherence, and stable grading behavior.

9 November 2025 in Informatics
Verified
Journal Article
Open access
PDF file
Catalin AnghelCatalin Anghel
Catalin Anghel
AAAndreea Alexandra Anghel
Andreea Alexandra Anghel
EPEmilia Pecheanu
Emilia Pecheanu
ACAdina Cocu
Adina Cocu
MCMarian Viorel Crăciun
Marian Viorel Crăciun
CAConstantin Adrian Andrei
Constantin Adrian Andrei
Background and objectives: Automated evaluation of open-ended responses remains a persistent challenge, particularly when consistency, transparency, and reproducibility are required. While large language models (LLMs) have shown promise in rubric-based evaluation, their reliability across multiple evaluators is still uncertain. Variability in scoring, feedback, and rubric adherence raises concerns about interpretability and system robustness. This study introduces GraderAssist, a graph-based, rubric-guided, multi-LLM framework designed to ensure transparent and reproducible automated evaluation. Methods: GraderAssist evaluates a dataset of 220 responses to both technical and argumentative questions, collected from undergraduate computer science courses. Six open-source LLMs and GPT-4 (as expert reference) independently scored each response using two predefined rubrics. All outputs—including scores, feedback, and metadata—were parsed, validated, and stored in a Neo4j graph database, enabling structured querying, traceability, and longitudinal analysis. Results: Cross-model analysis revealed systematic differences in scoring behavior and feedback generation. Some models produced more generous evaluations, while others aligned closely with GPT-4. Semantic analysis using Sentence-BERT embeddings highlighted distinctive feedback styles and variable rubric adherence. Inter-model agreement was stronger for technical criteria but diverged substantially for argumentative tasks. Originality: GraderAssist integrates rubric-guided evaluation, multi-model comparison, and graph-based storage into a unified pipeline. By emphasizing reproducibility, transparency, and fine-grained analysis of evaluator behavior, it advances the design of interpretable automated evaluation systems with applications in education and beyond.

3 December 2025 in Computers
Verified
Journal Article
Open access
PDF file
Catalin AnghelCatalin Anghel
Catalin Anghel
MCMarian Viorel Crăciun
Marian Viorel Crăciun
AAAndreea Alexandra Anghel
Andreea Alexandra Anghel
ACAdina Cocu
Adina Cocu
ABAntonio Stefan Balau
Antonio Stefan Balau
Cristian ScheauCristian Scheau
Cristian Scheau
Large Language Models (LLMs) are increasingly used for rubric-based assessment, yet reliability is limited by instability, bias, and weak diagnostics. We present EvalCouncil, a committee-and-chief framework for rubric-guided grading with auditable traces and a human adjudication baseline. Our objectives are to (i) characterize domain structure in Human–LLM alignment, (ii) assess robustness to concordance tolerance and panel composition, and (iii) derive a domain-adaptive audit policy grounded in dispersion and chief–panel differences. Authentic student responses from two domains–Computer Networks (CNs) and Machine Learning (ML)–are graded by multiple heterogeneous LLM evaluators using identical rubric prompts. A designated chief arbitrator operates within a tolerance band and issues the final grade. We quantify within-panel dispersion via MPAD (mean pairwise absolute deviation), measure chief–panel concordance (e.g., absolute error and bias), and compute Human–LLM deviation. Robustness is examined by sweeping the tolerance and performing leave-one-out perturbations of panel composition. All outputs and reasoning traces are stored in a graph database for full provenance. Human–LLM alignment exhibits systematic domain dependence: ML shows tighter central tendency and shorter upper tails, whereas CN displays broader dispersion with heavier upper tails and larger extreme spreads. Disagreement increases with item difficulty as captured by MPAD, concentrating misalignment on a relatively small subset of items. These patterns are stable to tolerance variation and single-grader removals. The signals support a practical triage policy: accept low-dispersion, small-gap items; apply a brief check to borderline cases; and adjudicate high-dispersion or large-gap items with targeted rubric clarification. EvalCouncil instantiates a committee-and-chief, rubric-guided grading workflow with committee arbitration, a human adjudication baseline, and graph-based auditability in a real classroom deployment. By linking domain-aware dispersion (MPAD), a policy tolerance dial, and chief–panel discrepancy, the study shows how these elements can be combined into a replicable, auditable, and capacity-aware approach for organizing LLM-assisted grading and identifying instability and systematic misalignment, while maintaining pedagogical interpretability.