Donggang Jia

Donggang Jia

Auto-generated profile, based on publicly available information

Donggang Jia

Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 June 2025 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Open access
DJDonggang Jia
Donggang Jia
AIAlexandra Irger
Alexandra Irger
LBLonni Besançon
Lonni Besançon
OSOndřej Strnad
Ondřej Strnad
DLDeng Luo
Deng Luo
IVIvan Viola
Ivan Viola
We present VOICE, a novel approach to science communication that connects large language models’ conversational capabilities with interactive exploratory visualization. VOICE introduces several innovative technical contributions that drive our conversational visualization framework. Based on the collected design requirements, we introduce a two-layer agent architecture that can perform task assignment, instruction extraction, and coherent content generation. We employ fine-tuning and prompt engineering techniques to tailor agents’ performance to their specific roles and accurately respond to user queries. Our interactive text-to-visualization method generates a flythrough sequence matching the content explanation. In addition, natural language interaction provides capabilities to navigate and manipulate 3D models in real-time. The VOICE framework can receive arbitrary voice commands from the user and respond verbally, tightly coupled with a corresponding visual representation, with low latency and high accuracy. We demonstrate the effectiveness of our approach by implementing a proof-of-concept prototype and applying it to the molecular visualization domain: analyzing three 3D molecular models with multiscale and multi-instance attributes. Finally, we conduct a comprehensive evaluation of the system, including quantitative and qualitative analyses on our collected dataset, along with a detailed public user study and expert interviews. The results confirm that our framework and prototype effectively meet the design requirements and cater to the needs of diverse target users.

14 May 2026 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
Dawar KhanDawar Khan
Dawar Khan
XLXinyu Liu
Xinyu Liu
OMOmar Mena
Omar Mena
DJDonggang Jia
Donggang Jia
AKAlexandre Kouyoumdjian
Alexandre Kouyoumdjian
IVIvan Viola
Ivan Viola
The deployment of large language models (LLMs) on extended reality (XR) devices has great potential to advance the field of human-AI interaction. In case of direct, on-device model inference, selecting the appropriate model and device for specific tasks remains challenging. In this paper, we present AIvaluateXR, a comprehensive evaluation framework for benchmarking LLMs running on XR devices. To demonstrate the framework, we deploy 17 selected LLMs across four XR platforms-Magic Leap 2, Meta Quest 3, Vivo X100s Pro, and Apple Vision Pro-and conduct an extensive evaluation. Our experimental setup measures four key metrics: performance consistency, processing speed, memory usage, and battery consumption. For each of the 68 model-device pairs, we assess performance under varying string lengths, batch sizes, and thread counts, analyzing the tradeoffs for real-time XR applications. We finally propose a unified evaluation method based on the 3D Pareto Optimality theory to select the optimal device-model pairs from the quality and speed objectives. Additionally, we compare the efficiency of on-device LLMs with client-server and cloud-based setups, and evaluate their accuracy on two interactive tasks. We believe our findings offer valuable insights to guide future optimization efforts for LLM deployment on XR devices. Our evaluation method can be followed as standard groundwork for further research and development in this emerging field.

14 December 2025
Verified
Conference Paper
Open access
DLDa Li
Da Li
DJDonggang Jia
Donggang Jia
YRYousef Rajeh
Yousef Rajeh
DEDominik Engel
Dominik Engel
IVIvan Viola
Ivan Viola

19 May 2026 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
DJDonggang Jia
Donggang Jia
YWYunhai Wang
Yunhai Wang
IVIvan Viola
Ivan Viola
Bioscientists frequently seek to visualize the biological systems they have empirically characterized and reported in the literature. Realizing such visualizations requires biological structure modeling, an inherently complex process that demands both biological and geometric understanding. This paper addresses the problem of constructing such 3D models for visualization. In this paper, we introduce a novel agent framework that mitigates the challenges of operating 3D modeling software by transforming user inputs, including natural language descriptions, research publication content, and textual descriptions of the existing objects and structures in the current scene, into modeling operations in a structured JSON format and final 3D results. The major technical contribution lies in the collaborative agent design that simultaneously supports model planning, execution, and novel user interaction design, such as interactive modeling execution and dynamic widget generation that fuse text and mouse interaction within the chat window. The framework further incorporates a customized modeling memory to enhance user interaction, featuring components such as personalized memory management, feedback collection, and skill library design. This modeling memory is leveraged to enable improved 3D modeling performance over time. The quantitative evaluation on our collected dataset showcases the effectiveness of our framework. We also develop a prototype tool, Chat Modeling, and demonstrate its usage through two modeling case studies. Our user study and expert interviews highlight the potential of our approach for use in scientific workflows.