27 September 2022 in IEEE Transactions on Visualization and Computer Graphics
Visual perception is a key component of data visualization. Much prior empirical work uses eye movement as a proxy to understand human visual perception. Diverse apparatus and techniques have been proposed to collect eye movements, but there is still no optimal approach. In this paper, we review 30 prior works for collecting eye movements based on three axes: (1) the tracker technology used to measure eye movements; (2) the image stimulus shown to participants; and (3) the collection methodology used to gather the data. Based on this taxonomy, we employ a webcam-based eyetracking approach using task-specific visualizations as the stimulus. The low technology requirement means that virtually anyone can participate, thus enabling us to collect data at large scale using crowdsourcing: approximately 12,000 samples in total. Choosing visualization images as stimulus means that the eye movements will be specific to perceptual tasks associated with visualization. We use these data to propose a SCANNER DEEPLY, a virtual eyetracker model that, given an image of a visualization, generates a gaze heatmap for that image. We employ a computationally efficient, yet powerful convolutional neural network for our model. We compare the results of our work with results from the DVS model and a neural network trained on the Salicon dataset. The analysis of our gaze patterns enables us to understand how users grasp the structure of visualized data. We also make our stimulus dataset of visualization images available as part of this paper's contribution.
13 June 2025 in IEEE Transactions on Visualization and Computer Graphics
Interactive visualization editors empower users to author visualizations without writing code, but do not provide guidance on the art and craft of effective visual communication. In this paper, we explore the potential of using an off-the-shelf large language models (LLMs) to provide actionable and customized feedback to visualization designers. Our implementation, Visualizationary, demonstrates how ChatGPT can be used for this purpose through two key components: a preamble of visualization design guidelines and a suite of perceptual filters that extract salient metrics from a visualization image. We present findings from a longitudinal user study involving 13 visualization designers—6 novices, 4 intermediates, and 3 experts—who authored a new visualization from scratch over several days. Our results indicate that providing guidance in natural language via an LLM can aid even seasoned designers in refining their visualizations. All our supplemental materials are available at https://osf.io/v7hu8.
21 November 2025 in IEEE Access
As artificial intelligence technology advances, the generalization performance of audio deepfake detection systems has improved rapidly and significantly. However, these systems still face substantial challenges when confronted with adversarial attacks. To address this issue, many studies have proposed defense methods against such attacks. Nevertheless, only a limited number of studies have focused on defenses in the context of audio deepfake detection. With the development of deep learning, deepfake detection methods are increasingly designed and deployed based on deep learning models. As a result, deepfake detection systems can be applied across various fields and are not restricted to specific data types or domains. Such approaches are referred to as ‘domain-independent’ defense methods and can be applied to a wide range of tasks, including adversarial training. However, they have limitations, such as requiring significant computing resources or potentially degrading original performance. In this paper, we propose Inference-Time Noise Addition (ITNA), an effective and efficient adversarial attack defense method that minimizes the original performance degradation. ITNA adds Gaussian noise once to the input audio sample during the inference phase. It is highly resource efficient, as it does not require additional training or separate AI components. Furthermore, ITNA minimizes performance degradation as much as possible and provides scalability that can be used with other defense methods. We provide a theoretical explanation of the effectiveness of ITNA and validate it through experimental results. Our experiments were conducted on the ASVspoof2021 DF evaluation (official deepfake audio dataset from the ASVspoof community), In-The-Wild (collected in real environments) and DSD-Corpus datasets (includes various synthesizer samples). And experiments results demonstrate the effectiveness of ITNA, reducing the average misclassification rate and the attack success rate of detection systems by 4.48% and 31.90%, respectively. We also demonstrate the effectiveness of ITNA through comparative experiments with alternative noise type and existing noise-based defense method.
5 December 2024 in Journal of Geophysical Research: Machine Learning and Computation
Neural networks (NNs) enable precise modeling of complicated geophysical phenomena but can be sensitive to small input changes. In this work, we present a new method for analyzing this instability in NNs. We focus our analysis on adversarial examples, test‐time inputs with carefully crafted human‐imperceptible perturbations that expose the worst‐case instability in a model's predictions. Our stability analysis is based on a low‐rank expansion of NNs on a fixed input, and we apply our analysis to a NN model for tsunami early warning which takes geodetic measurements as the input and forecasts tsunami waveforms. The result is an improved description of local stability that explains adversarial examples generated by a standard gradient‐based algorithm, and allows the generation of other comparable examples. Our analysis can predict whether noise in the geodetic input will produce an unstable output, and identifies a potential approach to filtering the input that enable more robust forecasting.