Background: The usability of university websites is important to ascertain that they serve their intended purpose. Their usability can be evaluated either by testing methods that rely on actual users or by inspection methods that rely on experts for evaluation. Heuristic evaluation and guideline reviews are two inspection methods of usability evaluation. A heuristic evaluation consists of a few general heuristics (rules), which are limited to checking general flaws in the design. A guideline review uses a much larger set of guidelines/suggestions that fit a specific business domain. Literature review: Most of the literature has equated usability studies with testing methods and has given less focus to inspection methods. Moreover, those studies have examined usability in a general sense and not in domain- and culture-specific contexts. Research questions: 1. Do domain- and culture-specific heuristic evaluation and guideline reviews work similarly in evaluating the usability of applications? 2. Which of these methods is better in terms of the nature of evaluation, time needed for evaluation, evaluation procedure, templates adopted, and evaluation results? 3. Which method is better in terms of thoroughness and reliability? Research methodology : This study uses a comparative methodology. The two inspection methods—guideline reviews and heuristic evaluation—are compared in a domain- and the culture-specific context in terms of the nature, time required, approach, templates, and results. Results: The results reflect that both methods identify similar usability issues; however, they differ in terms of the nature, time duration, evaluation procedure, templates, and results of the evaluation. Conclusion: This study contributes by providing insights for practitioners and researchers about the choice of an evaluation method for domain- and culture-specific evaluation of university websites.
20 March 2021 in Behaviour & Information Technology
Culture fit is an essential consideration while designing websites in this technologically evolving era, yet it is too often ignored in current usability evaluation methods. The heuristic evaluation is one such method, in which, generally, experts use a set of heuristics as a guide. These heuristics are well designed to pinpoint the general usability design faults; however, they limit identifying potential usability problems for specific domains and cultures. This article presents a formal methodology with a sequence of steps of how domain and culture-specific heuristics can be designed. This methodology is validated and applied in the context of websites for universities in Pakistan.
17 January 2026 in The Asian Bulletin of Big Data Management
Named Entity Recognition (NER) for low-resource languages remains challenging due to limited annotated corpora and complex linguistic characteristics. Urdu is a morphologically rich Indo-Aryan language written in a cursive right-to-left script, increases these challenges in contrast with high-resource languages in NER performance. This paper presents a data-efficient framework for Urdu NER that combines large-scale machine-generated annotations with high-quality human-annotated data to mitigate annotation scarcity. Firstly, a new gold-standard human-annotated Urdu NER corpus is built comprising 49,040 tokens and 5,839 named entities across 13 fine-grained entity categories, including newly introduced types such as Sports, Food, and Color. To complement this dataset, a large machine-annotated corpus was created using a bootstrapped ensemble of Conditional Random Field models with confidence-based filtering. A two-stage training strategy in which multilingual transformer models, Multilingual BERT (mBERT) and XLM-RoBERTa (XLM-R) are pre-trained on the machine-annotated corpus and subsequently fine-tuned on the human-annotated data. Experimental results demonstrate that machine-annotation pre-training consistently improves NER performance for both models, yielding micro-averaged F1-score gains from 0.84 to 0.86 for mBERT and from 0.86 to 0.88 for XLM-R. Detailed per-class and confusion matrix analyses further show notable improvements for low-frequency and specialized entity types. The findings confirm that integrating weakly supervised machine annotation with multilingual transformer-based learning provides a practical and scalable solution for improving NER in Urdu and other low-resource languages. Keywords: NER, machine-annotated corpus, low-resource languages, Urdu NER corpus
Usability evaluations are essential for ensuring that modern interfaces meet user needs, yet traditional heuristic evaluations by human experts can be time-consuming and subjective, especially early in development. This paper investigates whether large language models (LLMs) can provide reliable and consistent heuristic assessments at the development stage. By applying Jakob Nielsen's ten usability heuristics to thirty open-source websites, we generated over 850 heuristic evaluations in three independent evaluations per site using a pipeline of OpenAI's GPT-4o. For issue detection, the model demonstrated moderate consistency, with an average pairwise Cohen's Kappa of 0.50 and an exact agreement of 84%. Severity judgments showed more variability: weighted Cohen's Kappa averaged 0.63, but exact agreement was just 56%, and Krippendorff's Alpha was near zero. These results suggest that while GPT-4o can produce internally consistent evaluations, especially for identifying the presence of usability issues, its ability to judge severity varies and requires human oversight in practice. Our findings highlight the feasibility and limitations of using LLMs for early-stage, automated usability testing, and offer a foundation for improving consistency in automated User Experience (UX) evaluation. To the best of our knowledge, our work provides one of the first quantitative inter-rater reliability analyses of automated heuristic evaluation and highlights methods for improving model consistency.