2 June 2025 in IEEE Access
Transformer-based video models have performed SOTA in various action recognition and video understanding tasks. The limitation is the lack of explainability of these complex models. Current Explainable AI (XAI) methods focus on feature importance analysis of only one dimension, either spatial or temporal features. When applied to transformer-based video models, the challenges are two-fold. They fail to capture the integrated spatio-temporal nature of video data and incur prohibitive computational costs for real-time applications. This paper presents STAA (Spatio-Temporal Attention Attribution), an XAI method for interpreting video transformer models for action recognition tasks. STAA simultaneously extracts both spatial importance within frames and temporal relevance across the video sequence from attention values in transformers. This unified approach solves the challenge by capturing how transformer models integrate information across both dimensions for decision-making, providing comprehensive explanations that reflect the model’s actual reasoning process. We enhance STAA’s raw output through post-processing. The experiments on the Kinetics-400 dataset demonstrate superior faithfulness (0.844 ± 0.116) and monotonicity (0.850 ± 0.030). In terms of computational overhead, our STAA method is applicable for real-time video XAI analysis applications. We implement a cloud-based architecture that enables video explanations with 150 ms latency.
10 April 2023 in Computer
In this article, we follow a process of explainable artificial intelligence (XAI) method development and define two metrics in terms of consistency and efficiency in guiding the evaluation of XAI explanations.
10 May 2024 in IEEE Transactions on Cloud Computing
This paper presents the design of an open-API-based explainable AI (XAI) service to provide feature contribution explanations for cloud AI services. Cloud AI services are widely used to develop domain-specific applications with precise learning metrics. However, the underlying cloud AI services remain opaque on how the model produces the prediction. We argue that XAI operations are accessible as open APIs to enable the consolidation of the XAI operations into the cloud AI services assessment. We propose a design using a microservice architecture that offers feature contribution explanations for cloud AI services without unfolding the network structure of the cloud models. We can also utilize this architecture to evaluate the model performance and XAI consistency metrics showing cloud AI services' trustworthiness. We collect provenance data from operational pipelines to enable reproducibility within the XAI service. Furthermore, we present the discovery scenarios for the experimental tests regarding model performance and XAI consistency metrics for the leading cloud vision AI services. The results confirm that the architecture, based on open APIs, is cloud-agnostic. Additionally, data augmentations result in measurable improvements in XAI consistency metrics for cloud AI services.
5 September 2025 in ACM Transactions on Multimedia Computing, Communications, and Applications
The widespread deployment of video transformer models in action recognition systems necessitates a comprehensive understanding of their vulnerability to adversarial attacks. Unlike traditional CNN-based video models, transformers process spatiotemporal dependencies through self-attention mechanisms, creating a different vulnerability profile to adversarial attacks. This study presents an investigation of adversarial robustness in video transformers. We develop a novel joint spatiotemporal attack method that precisely targets the attention mechanisms of video transformers. By simultaneously perturbing both spatial and temporal features, our method achieves a 76.30% in ASR on the Kinetics-400 dataset, outperforming frame-wise attacks and state-of-the-art query-based attacks. To interpret the mechanisms underlying these attacks, we introduce quantitative metrics based on Explainable AI (XAI) analysis. Spatial analysis reveals systematic disruption of attention patterns, with adversarial examples showing median SSIM scores of 0.353. Temporal correlation analysis also demonstrates severe degradation in attention coherence across frame sequences. Through experiments comparing previous attack methods, including common corruptions benchmark, frame-wise attacks, sparse attacks, and recent V-BAD attacks, we demonstrate that our proposed method is more effective in transformer-based video models. This study further examines the adversarial training strategy against the selected attacks. To promote reproducibility and facilitate future research, we provide our methods and analysis tools through a public GitHub repository. These findings underscore the effectiveness of jointly considering spatial and temporal dimensions when developing adversarial attack strategies and defense mechanisms for video AI models.
22 December 2023 in TechRxiv
This paper presents the design of an open-API-based explainable AI (XAI) service to provide feature-contribution explanations for cloud AI services. Cloud AI services have broad usage in developing domain-specific applications with learning precision metrics. However, the underlying AI models remain opaque on how the model produces the prediction. We argue that XAI operations are accessible as open APIs to enable the consolidation of the XAI operations into the cloud AI services assessment. We propose a design using a microservice architecture that offers feature contribution explanations for cloud AI services without unfolding the network structure of the cloud AI services. We can also utilize this architecture to evaluate the performance and XAI consistency metrics showing cloud AI services' trustworthiness. We collect provenance data from XAI operations to enable traceability within the XAI service. Furthermore, we present the discovery scenarios for the experimental tests regarding performance and XAI consistency metrics for the leading cloud AI services for computer vision. The results confirm the open-API-based architecture cloud-agnostic. Additionally, data augmentation has a marked improvement in XAI consistency metrics for the cloud AI services.