The proliferation of 3D representations, from explicit meshes to implicit neural fields and more, motivates the need for simulators agnostic to representation. We present a data-, mesh-, and grid-free solution for elastic simulation for any object in any geometric representation undergoing large, nonlinear deformations. We note that every standard geometric representation can be reduced to an occupancy function queried at any point in space, and we define a simulator atop this common interface. For each object, we fit a small implicit neural network encoding spatially varying weights that act as a reduced deformation basis. These weights are trained to learn physically significant motions in the object via random perturbations. Our loss ensures we find a weight-space basis that best minimizes deformation energy by stochastically evaluating elastic energies through Monte Carlo sampling of the deformation volume. At runtime, we simulate in the reduced basis and sample the deformations back to the original domain. Our experiments demonstrate the versatility, accuracy, and speed of this approach on data including signed distance functions, point clouds, neural primitives, tomography scans, radiance fields, Gaussian splats, surface meshes, and volume meshes, as well as showing a variety of material energies, contact models, and time integration schemes.

In this paper, we investigate blended physical-digital kinesthetic feedback (or blended haptics in short) as a means for controlled three-dimensional design ideation in mixed reality (MR) environments. We define blended haptics as a spatial interaction wherein a physical object (e.g. a 3D printed human head) provides a specific design context for a user to generate ideas through physical manipulation of and on the object (e.g. drawing a digital sketch of a helmet) in the virtual environment. Using 3D wire-frame modeling as a concrete digital prototyping context, we investigate this idea of blended haptics in terms of how it supports design cognition specifically in spatial user interfaces. For this, we implemented a modeling tool as an experimental setup that allows a user to directly create curve-networks (wire-frame models) on a physical object (i.e. a contextual proxy) with one hand while simultaneously controlling the object with the other hand using a tracked turn-table. The key idea is that the user can simultaneously experience kinesthetic feedback from both the physical objects as well as the digital wire-frame models. To systematically investigate our approach, we conducted a comparative user evaluation of conceptual design tasks performed by two groups of users, one with the blended haptics (e.g. physical head and digital helmet) and the other with purely digital haptic feedback (e.g. digital head and digital helmet). Our study shows that blended haptics required less physical effort in the design task and resulted in concepts with higher novelty score as compared to using purely digital haptic feedback.

We present neural collision fields as an alternative to contact point sampling in physics simulations. Our approach is built on top of a novel smoothed integral formulation for the contact surface patches between two triangle meshes. By reformulating collisions as an integral, we avoid issues of sampling common to many collision-handling algorithms. Because the resulting integral is difficult to evaluate numerically, we store its solution in an integrated neural collision field — a 6D neural field in the space of triangle pair vertex coordinates. Our network generalizes well to new triangle meshes without retraining. We demonstrate the effectiveness of our method by implementing it as a constraint in a position-based dynamics framework and show that our neural formulation successfully handles collisions in practical simulations involving both volumetric and thin-shell geometries.

We propose a learning‐based approach for full‐body pose reconstruction from extremely sparse upper body tracking data, obtained from a virtual reality (VR) device. We leverage a conditional variational autoencoder with gated recurrent units to synthesize plausible and temporally coherent motions from 4‐point tracking (head, hands, and waist positions and orientations). To avoid synthesizing implausible poses, we propose a novel sample selection and interpolation strategy along with an anomaly detection algorithm. Specifically, we monitor the quality of our generated poses using the anomaly detection algorithm and smoothly transition to better samples when the quality falls below a statistically defined threshold. Moreover, we demonstrate that our sample selection and interpolation method can be used for other applications, such as target hitting and collision avoidance, where the generated motions should adhere to the constraints of the virtual environment. Our system is lightweight, operates in real‐time, and is able to produce temporally coherent and realistic motions.