Creativity Support Tools (CSTs) aid in the efficient and effective composition of creative content, such as picture books. However, many existing CSTs only allow for mono-modal creation, whereas previous studies have become theoretically and technically mature to support multi-modal innovative creations. To overcome this limitation, we introduce XCreation, a novel CST that leverages generative AI to support cross-modal storybook creation. Nevertheless, directly deploying AI models to CSTs can still be problematic as they are mostly black-box architectures that are not comprehensible to human users. Therefore, we integrate an interpretable entity-relation graph to intuitively represent picture elements and their relations, improving the usability of the underlying generative structures. Our between-subject user study demonstrates that XCreation supports continuous plot creation with increased creativity, controllability, usability, and interpretability. XCreation is applicable to various scenarios, including interactive storytelling and picture book creation, thanks to its multimodal nature.
This paper tackles the problem of creating relightable and animatable neural avatars from sparse-view (or monocular) videos of dynamic humans under unknown illumination. Previous neural human reconstruction methods produce animatable avatars from sparse views using deformed Signed Distance Fields (SDF) but are non-relightable. While differentiable inverse rendering methods have succeeded in the material recovery of static objects, it is not straightforward to extend them to dynamic humans since it is computationally intensive to compute pixel-surface intersection and light visibility on deformed SDFs for relighting. To solve this challenge, we propose a Hierarchical Distance Query (HDQ) algorithm to approximate the world space SDF under arbitrary human poses. Specifically, we estimate coarse SDF based on a parametric human model and compute fine SDF by exploiting the invariance of SDF w.r.t. local deformation. Based on HDQ, we leverage sphere tracing to efficiently estimate the surface intersection and light visibility. This allows us to develop the first system to recover relightable and animatable neural avatars from sparse or monocular inputs. Experiments show that our approach produces superior results compared to state-of-the-art methods. Our project page is available at https://zju3dv.github.io/relightable_avatar.
1 January 2021 in Therapeutic Advances in Neurological Disorders
The role of statins in unruptured intracranial aneurysm (UIA) growth and rupture remains ambiguous. This study sought to determine whether atorvastatin is associated with aneurysm growth and rupture in patients harboring UIA <7 mm. This prospective, multicenter cohort study consecutively enrolled patients with concurrent UIA <7 mm and ischemic cerebrovascular disease from four hospitals between 2016 and 2019. Baseline and follow-up patient information was recorded. Because of the strong anti-inflammatory effect of aspirin, patients using aspirin were excluded. Patients taking atorvastatin 20 mg daily were atorvastatin users. The primary and exploratory endpoints were aneurysm rupture and growth, respectively. Among the 1087 enrolled patients, 489 (45.0%) took atorvastatin, and 598 (55%) took no atorvastatin. After a mean follow-up duration of 33.0 ± 12.5 months, six (1.2%) and five (0.8%) aneurysms ruptured in atorvastatin and non-atorvastatin groups, respectively. In the adjusted multivariate Cox analysis, UIA sized 5 to <7 mm, current smoker, and uncontrolled hypertension were associated with aneurysm rupture, whereas atorvastatin [adjusted hazard ratio (HR) 1.495, 95% confidence interval (CI) 0.417–5.356, p = 0.537] was not. Of 159 patients who had follow-up imaging, 34 (21.4%) took atorvastatin and 125 (78.6%) took no atorvastatin. Aneurysm growth occurred in five (14.7%) and 21 (16.8%) patients in atorvastatin and non-atorvastatin groups (mean follow-up: 20.2 ± 12.9 months), respectively. In the adjusted multivariate Cox analysis, UIAs sized 5 to <7 mm and uncontrolled hypertension were associated with a high growth rate; atorvastatin (adjusted HR 0.151, 95% CI 0.031–0.729, p = 0.019) was associated with a reduced growth rate. We conclude atorvastatin use is associated with a reduced risk of UIA growth, whereas atorvastatin is not associated with UIA rupture.
27 March 2023 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Smart orthoses hold great potential for intelligent rehabilitation monitoring and training. However, most of these electronic assistive devices are typically too difficult for daily use and challenging to modify to accommodate variations in body shape and medical needs. For existing clinicians, the customization pipeline of these smart devices imposes significant learning costs. This paper introduces ThermoFit, an end-to-end design and fabrication pipeline for thermoforming smart orthoses that adheres to the clinically accepted procedure. ThermoFit enables the shapes and electronics positions of smart orthoses to conform to bodies and allows rapid iteration by integrating low-cost Low-Temperature Thermoplastics (LTTPs) with custom metamaterial structures and electronic components. Specifically, three types of metamaterial structures are used in LTTPs to reduce the wrinkles caused by the thermoforming process and to permit component position adjustment and joint movement. A design tool prototype aids in generating metamaterial patterns and optimizing component placement and circuit routing. Three applications show that ThermoFit can be shaped on bodies to different wearables. Finally, a hands-on study with a clinician verifies the user-friendliness of thermoforming smart orthosis, and technical evaluations demonstrate fabrication efficiency and electronic continuity.
5 October 2022 in IEEE Transactions on Visualization and Computer Graphics
In history research, cohort analysis seeks to identify social structures and figure mobilities by studying the group-based behavior of historical figures. Prior works mainly employ automatic data mining approaches, lacking effective visual explanation. In this paper, we present CohortVA, an interactive visual analytic approach that enables historians to incorporate expertise and insight into the iterative exploration process. The kernel of CohortVA is a novel identification model that generates candidate cohorts and constructs cohort features by means of pre-built knowledge graphs constructed from large-scale history databases. We propose a set of coordinated views to illustrate identified cohorts and features coupled with historical events and figure profiles. Two case studies and interviews with historians demonstrate that CohortVA can greatly enhance the capabilities of cohort identifications, figure authentications, and hypothesis generation.
27 September 2023 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Nod and shake of one's head are intuitive and universal gestures in communication. As smartwatches become increasingly intelligent through advances in user activity sensing technologies, many use scenarios of smartwatches demand quick responses from users in confirmation dialogs, to accept or dismiss proposed actions. Such proposed actions include making emergency calls, taking service recommendations, and starting or stopping exercise timers. Head gestures in these scenarios could be preferable to touch interactions for being hands-free and easy to perform. We propose Headar to recognize these gestures on smartwatches using wearable millimeter wave sensing. We first surveyed head gestures to understand how they are performed in conversational settings. We then investigated positions and orientations to which users raise their smartwatches. Insights from these studies guided the implementation of Headar. Additionally, we conducted modeling and simulation to verify our sensing principle. We developed a real-time sensing and inference pipeline using contemporary deep learning techniques, and proved the feasibility of our proposed approach with a user study (n=15) and a live test (n=8). Our evaluation yielded an average accuracy of 84.0% in the user study across 9 classes including nod and shake as well as seven other signals -- still, speech, touch interaction, and four non-gestural head motions (i.e., head up, left, right, and down). Furthermore, we obtained an accuracy of 72.6% in the live test which reveals rich insights into the performance of our approach in various realistic conditions.