Mixed Reality (MR) systems display content freely in space, and present nearly arbitrary amounts of information, enabling ubiquitous access to digital information. This approach, however, introduces clutter and distraction if too much virtual content is shown. We present BlendMR, an optimization-based MR system that blends virtual content onto the physical objects in users’ environments to serve as ambient information displays. Our approach takes existing 2D applications and meshes of physical objects as input. It analyses the geometry of the physical objects and identifies regions that are suitable hosts for virtual elements. Using a novel integer programming formulation, our approach then optimally maps selected contents of the 2D applications onto the object, optimizing for factors such as importance and hierarchy of information, viewing angle, and geometric distortion. We evaluate BlendMR by comparing it to a 2D window baseline. Study results show that BlendMR decreases clutter and distraction, and is preferred by users. We demonstrate the applicability of BlendMR in a series of results and usage scenarios.
We propose augmenting initially passive structures built from simple repeated cells, with novel active units to enable dynamic, shape-changing, and robotic applications. Inspired by metamaterials that can employ mechanisms, we build a framework that allows users to configure cells of this passive structure to allow it to perform complex tasks. A key benefit is that our structures can be repeatedly (re)configured by users inserting our configuration units to turn the passive material into, e.g., locomotion robots, integrated motion platforms, or interactive interfaces, as we demonstrate in this paper. To this end, we present a mechanical system consisting of a flexible, passive, shearing lattice structure, as well as rigid and active unit cells to be inserted into the lattice for configuration. The active unit is a closed-loop pneumatically controlled shearing cell to dynamically actuate the macroscopic movement of the structure. The passive rigid cells redirect the forces to create complex motion with a reduced number of active cells. Since the placement of the rigid and active units is challenging, we offer a computational design tool. The tool optimizes the cell placement to match the macroscopic, user-defined target motions and generates the control code for the active cells.