Ken Museth

Ken Museth

Auto-generated profile, based on publicly available information

Ken Museth

NVIDIA, Santa Clara, USA

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 February 2024 in ACM Transactions on Graphics
Verified
Journal Article
Open access
DKDoyub Kim
Doyub Kim
MLMinjae Lee
Minjae Lee
KMKen Museth
Ken Museth
We introduce NeuralVDB, which improves on an existing industry standard for efficient storage of sparse volumetric data, denoted VDB [Museth2013], by leveraging recent advancements in machine learning. Our novel hybrid data structure can reduce the memory footprints of VDB volumes by orders of magnitude, while maintaining its flexibility and only incurring small (user-controlled) compression errors. Specifically, NeuralVDB replaces the lower nodes of a shallow and wide VDB tree structure with multiple hierarchical neural networks that separately encode topology and value information by means of neural classifiers and regressors respectively. This approach is proven to maximize the compression ratio while maintaining the spatial adaptivity offered by the higher-level VDB data structure. For sparse signed distance fields and density volumes, we have observed compression ratios on the order of 10× to more than 100× from already compressed VDB inputs, with little to no visual artifacts. Furthermore, NeuralVDB is shown to offer more effective compression performance compared to other neural representations such as Neural Geometric Level of Detail [Takikawa et al.2021], Variable Bitrate Neural Fields [Takikawa et al.2022a], and Instant Neural Graphics Primitives [Müller et al.2022]. Finally, we demonstrate how warm-starting from previous frames can accelerate training, i.e., compression, of animated volumes as well as improve temporal coherency of model inference, i.e., decompression.

19 July 2024 in ACM Transactions on Graphics
Verified
Journal Article
Open access
FWFrancis Williams
Francis Williams
JHJiahui Huang
Jiahui Huang
JSJonathan Swartz
Jonathan Swartz
GKGergely Klar
Gergely Klar
VTVijay Thakkar
Vijay Thakkar
KMKen Museth
Ken Museth
We present f VDB, a novel GPU-optimized framework for deep learning on large-scale 3D data. f VDB provides a complete set of differentiable primitives to build deep learning architectures for common tasks in 3D learning such as convolution, pooling, attention, ray-tracing, meshing, etc. f VDB simultaneously provides a much larger feature set (primitives and operators) than established frameworks with no loss in efficiency: our operators match or exceed the performance of other frameworks with narrower scope. Furthermore, f VDB can process datasets with much larger footprint and spatial resolution than prior works, while providing a competitive memory footprint on small inputs. To achieve this combination of versatility and performance, f VDB relies on a single novel VDB index grid acceleration structure paired with several key innovations including GPU accelerated sparse grid construction, convolution using tensorcores, fast ray tracing kernels using a Hierarchical Digital Differential Analyzer algorithm (HDDA), and jagged tensors. Our framework is fully integrated with PyTorch enabling interoperability with existing pipelines, and we demonstrate its effectiveness on a number of representative tasks such as large-scale point-cloud segmentation, high resolution 3D generative modeling, unbounded scale Neural Radiance Fields, and large-scale point cloud reconstruction.

9 August 2024 in ACM Transactions on Graphics
Verified
Journal Article
Open access
HPHyojoon Park
Hyojoon Park
SSSangeetha Grama Srinivasan
Sangeetha Grama Srinivasan
MCMatthew Cong
Matthew Cong
DKDoyub Kim
Doyub Kim
BKByungsoo Kim
Byungsoo Kim
ESEftychios Sifakis
Eftychios Sifakis
We present a neural network-based simulation super-resolution framework that can efficiently and realistically enhance a facial performance produced by a low-cost, real-time physics-based simulation to a level of detail that closely approximates that of a reference-quality off-line simulator with much higher resolution (27× element count in our examples) and accurate physical modeling. Our approach is rooted in our ability to construct a training set of paired frames, from the low- and high-resolution simulators respectively, that are in semantic correspondence with each other. We use face animation as an exemplar of such a simulation domain, where creating this semantic congruence is achieved by simply dialing in the same muscle actuation controls and skeletal pose in the two simulators. Our proposed neural network super-resolution framework generalizes from this training set to unseen expressions, compensates for modeling discrepancies between the two simulations due to limited resolution or cost-cutting approximations in the real-time variant, and does not require any semantic descriptors or parameters to be provided as input, other than the result of the real-time simulation. We evaluate the efficacy of our pipeline on a variety of expressive performances and provide comparisons and ablation experiments for plausible variations and alternatives to our proposed scheme. Our code is available at https://github.com/hjoonpark/3d-sim-super- res.git.

28 July 2019
Verified
Conference Paper
Open access
KMKen Museth
Ken Museth
NANick Avramoussis
Nick Avramoussis
DBDan Bailey
Dan Bailey

27 July 2025
Verified
Conference Paper
Open access
AMAbhishek Madan
Abhishek Madan
NSNicholas Sharp
Nicholas Sharp
FWFrancis Williams
Francis Williams
KMKen Museth
Ken Museth
DLDavid I.W. Levin
David I.W. Levin

6 August 2023
Verified
Conference Paper
Open access
GKGergely Klar
Gergely Klar
SWStephen Ward
Stephen Ward
AMAndrew Moffat
Andrew Moffat
ESEftychios Sifakis
Eftychios Sifakis
KMKen Museth
Ken Museth
We present a volumetric, simulation-based pipeline for the automatic creation of strain-based descriptors from facial performance capture provided as surface meshes. Strain descriptors encode facial poses via length elongation/contraction ratios of curves embedded in the flesh volume. Strains are anatomically motivated, correlate strongly to muscle action, and offer excellent coverage of the pose space. Our proposed framework extracts such descriptors from surface-only performance capture data, by extrapolating this deformation into the flesh volume in a physics-based fashion that respects collisions and filters non-physical capture defects. The result of our system feeds into Machine Learning facial animation tools, as employed in Avatar: The Way of Water.