Jaron Lanier

Jaron Lanier

Auto-generated profile, based on publicly available information

Jaron Lanier

Microsoft, United States

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 May 2024
Verified
Conference Paper
Open access
FTFernanda De La Torre
Fernanda De La Torre
CFCathy Mengying Fang
Cathy Mengying Fang
HHHan Huang
Han Huang
ABAndrzej Banburski-Fahey
Andrzej Banburski-Fahey
JFJudith Amores Fernandez
Judith Amores Fernandez
JLJaron Lanier
Jaron Lanier
We present Large Language Model for Mixed Reality (LLMR), a framework for the real-time creation and modification of interactive Mixed Reality experiences using LLMs. LLMR leverages novel strategies to tackle difficult cases where ideal training data is scarce, or where the design goal requires the synthesis of internal dynamics, intuitive analysis, or advanced interactivity. Our framework relies on text interaction and the Unity game engine. By incorporating techniques for scene understanding, task planning, self-debugging, and memory management, LLMR outperforms the standard GPT-4 by 4x in average error rate. We demonstrate LLMR’s cross-platform interoperability with several example worlds, and evaluate it on a variety of creation and modification tasks to show that it can produce and edit diverse objects, tools, and scenes. Finally, we conducted a usability study (N=11) with a diverse set that revealed participants had positive experiences with the system and would use it again.

10 July 2023
Verified
Conference Paper
Open access
PPPayod Panda
Payod Panda
MNMolly Jane Nicholas
Molly Jane Nicholas
DNDavid Nguyen
David Nguyen
EOEyal Ofek
Eyal Ofek
MPMichel Pahud
Michel Pahud
JLJaron Lanier
Jaron Lanier
Via Research through Design (RtD), we explore the potential of headphones as a general-purpose input device for both foreground motion-gestures as well as background sensing of user activity. As a familiar wearable device, headphones offer a compelling site for head-situated interaction and sensing. Using emerging sensing modalities such as inertial motion, capacitive touch sensing, and depth cameras, our implemented prototypes explore sensing and interaction techniques that offer a range of compelling capabilities. User scenarios include context-aware privacy, gestural audio-visual control, and co-opting natural body language as context to drive animated avatars for "camera-off" scenarios in remote work–or to co-opt (oft-subconscious) head movements such as dodging attacks in video games to enhance the gameplay experience. Drawing from literature and other frameworks, we situate our prototypes and related techniques in a design space across the dual dimensions of (1) type of input (touch, mid-air, or head orientation); and (2) the context of user action (application, body, or environment). In particular, interactions that combine multiple inputs and contexts at the same time offer a rich design space of headphone-situated wearable interactions and sensing techniques.

12 November 2025
Verified
Conference Paper
Open access
AKAmina Kobenova
Amina Kobenova
CDCyan DeVeaux
Cyan DeVeaux
SPSamyak Parajuli
Samyak Parajuli
ABAndrzej Banburski-Fahey
Andrzej Banburski-Fahey
JAJudith Amores
Judith Amores
JLJaron Lanier
Jaron Lanier

1 September 2016
Verified
Conference Paper
Closed access
JLJaron Lanier
Jaron Lanier
VMVictor Mateevitsi
Victor Mateevitsi
KRKishore Rathinavel
Kishore Rathinavel
LSLior Shapira
Lior Shapira
JMJoseph Menke
Joseph Menke
WCWayne Chang
Wayne Chang
Optical see-through (OST) displays can overlay computer generated graphics on top of the physical world, effectually fusing the two worlds together. However, current OST displays have a limited (compared to the human) field-of-view (FOV) and are powered by laptops which hinders their mobility. Furthermore the systems are designed for single-user experiences and therefore cannot be used for collocated multi-user applications. In this paper we contribute the design of the RealityMashers, two wide FOV OST displays that can be manufactured using rapid-prototyping techniques. We also contribute preliminary user feedback providing insights into enhancing future RealityMasher experiences. By providing the RealityMasher's schematics we hope to make Augmented Reality more accessible and as a result accelerate the research in the field.