David Luebke

David Luebke

Auto-generated profile, based on publicly available information

David Luebke

NVIDIA

Research Focus & Metrics

Academic Career Length

24Years (2003-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 July 2023
Verified
Conference Paper
Open access
MSMichael Stengel
Michael Stengel
KNKoki Nagano
Koki Nagano
CLChao Liu
Chao Liu
MCMatthew Chan
Matthew Chan
ATAlex Trevithick
Alex Trevithick
DLDavid Luebke
David Luebke
We present an AI-mediated 3D video conferencing system that can reconstruct and autostereoscopically display a life-sized talking head using consumer-grade compute resources and minimal capture equipment. Our 3D capture uses a novel 3D lifting method that encodes a given 2D input into an efficient triplanar neural representation of the user, which can be rendered from novel viewpoints in real-time. Our AI-based techniques drastically reduce the cost for 3D capture, while providing a high-fidelity 3D representation on the receiver’s end at the cost of traditional 2D video streaming. Additional advantages of our AI-based approach include the ability to accommodate both photorealistic and stylized avatars, and the ability to enable mutual eye contact in multi-directional video conferencing. We demonstrate our system using a tracked stereo display for a personal viewing experience as well as a lightfield display for a room-scale multi-viewer experience.

22 November 2024
Verified
Book Chapter
Closed access
PDF file
EPEkta Prashnani
Ekta Prashnani
KNKoki Nagano
Koki Nagano
SMShalini De Mello
Shalini De Mello
DLDavid Luebke
David Luebke
OGOrazio Gallo
Orazio Gallo

11 January 2023 in ACM Transactions on Applied Perception
Verified
Journal Article
Open access
RBRachel Brown
Rachel Brown
VDVasha Dutell
Vasha Dutell
BWBruce Walter
Bruce Walter
RRRuth Rosenholtz
Ruth Rosenholtz
PSPeter Shirley
Peter Shirley
DLDavid Luebke
David Luebke
Computer graphics seeks to deliver compelling images, generated within a computing budget, targeted at a specific display device, and ultimately viewed by an individual user. The foveated nature of human vision offers an opportunity to efficiently allocate computation and compression to appropriate areas of the viewer’s visual field, of particular importance with the rise of high-resolution and wide field-of-view display devices. However, while variations in acuity and contrast sensitivity across the field of view have been well-studied and modeled, a more consequential variation concerns peripheral vision’s degradation in the face of clutter, known as crowding. Understanding of peripheral crowding has greatly advanced in recent years, in terms of both phenomenology and modeling. Accurately leveraging this knowledge is critical for many applications, as peripheral vision covers a majority of pixels in the image. We advance computational models for peripheral vision aimed toward their eventual use in computer graphics. In particular, researchers have recently developed high-performing models of peripheral crowding, known as “pooling” models, which predict a wide range of phenomena but are computationally inefficient. We reformulate the problem as a dataflow computation, which enables faster processing and operating on larger images. Further, we account for the explicit encoding of “end stopped” features in the image, which was missing from previous methods. We evaluate our model in the context of perception of textures in the periphery, including a novel texture dataset and updated textural descriptors. Our improved computational framework may simplify development and testing of more sophisticated, complete models in more robust and realistic settings relevant to computer graphics.


6 September 2019 in Journal of Vision
Verified
Journal Article
Open access
LZLili Zhang
Lili Zhang
RARachel Albert
Rachel Albert
JKJoohwan Kim
Joohwan Kim
DLDavid Luebke
David Luebke
Gaze-contingent rendering (also called foveated rendering) is a technique for increasing rendering efficiency by displaying reduced-fidelity content outside the fixation region. It has the potential to lower computational costs as well as reduce bandwidth and latency for cloud-based rendering. Color discrimination is known to be degraded in the periphery due to photoreceptor distribution (Curcio & Allen, 1990) and cortical magnification (Abramov et al., 1991), suggesting a potential for significant savings. However, an eccentricity-dependent color discrimination model is required to reduce peripheral color accuracy without detection. We built a model describing peripheral color difference tolerances, adapted from the CIEDE2000 color difference model. We used eccentricity-dependent parameters for hue and chroma based on measured peripheral chromatic discrimination thresholds from Hansen et al., 2009. We conducted two experiments to test our model. First, we compared the predicted versus actual thresholds by testing multiple levels of CIELAB chroma and direction (increased or decreased chroma) on three image types (simple, vector, and natural). Second, we validated the utility of the model as a visual difference predictor (VDP) for per-channel bit reduction of natural images (peripheral images were rendered at lower bit depth). In both experiments, subjects were asked to freely view high-resolution static images with real-time eye tracking and peripheral color degradation, responding whether they noticed any artifacts. Results indicate the model slightly overestimates color difference thresholds for some subjects and image types. There is a strong trend of content dependency, with more complex images producing higher and more consistent thresholds across subjects. No difference was found between chroma directions. Our simple model shows some predictive power as a VDP for gaze-contingent color degradation. However, additional perceptual effects such as chromatic crowding and peripheral spatial frequency characteristics would likely produce more accurate results. Further study is required for practical applications.

1 January 2003
Verified
Book Chapter
Closed access
DLDavid Luebke
David Luebke
MRMartin Reddy
Martin Reddy
Jonathan D. CohenJonathan D. Cohen
Jonathan D. Cohen
AVAmitabh Varshney
Amitabh Varshney
BWBenjamin Watson
Benjamin Watson
RHRobert Huebner
Robert Huebner