Ultrasound manipulation is growing in popularity in the HCI community with applications in haptics, on-body interaction, and levitation-based displays. Most of these applications share two key limitations: a) the complexity of the sound fields that can be produced is limited by the physical size of the transducers, and b) no obstacles can be present between the transducers and the control point. We present SoundBender, a hybrid system that overcomes these limitations by combining the versatility of phased arrays of Transducers (PATs) with the precision of acoustic metamaterials. In this paper, we explain our approach to design and implement such hybrid modulators (i.e. to create complex sound fields) and methods to manipulate the field dynamically (i.e. stretch, steer). We demonstrate our concept using self-bending beams enabling both levitation and tactile feedback around an obstacle and present example applications enabled by SoundBender.

We present Tangible Drops, a visio-tactile display that for the first time provides physical visualization and tactile feedback using a planar liquid interface. It presents digital information interactively by tracing dynamic patterns on horizontal flat surfaces using liquid metal drops on a programmable electrode array. It provides tactile feedback with directional information in the 2D vector plane using linear locomotion and/or vibration of the liquid metal drops. We demonstrate move, oscillate, merge, split and dispense-from-reservoir functions of the liquid metal drops by consuming low power (450 mW per electrode) and low voltage (8--15 V). We report on results of our empirical study with 12 participants on tactile feedback using 8 mm diameter drops, which indicate that Tangible Drops can convey tactile sensations such as changing speed, varying direction and controlled oscillation with no visual feedback. We present the design space and demonstrate the applications of Tangible Drops, and conclude by suggesting potential future applications for the technique.

We present MistForm, a shape changing fog display that can support one or two users interacting with either 2D or 3D content. Mistform combines affordances from both shape changing interfaces and mid-air displays. For example, a concave display can maintain content in comfortable reach for a single user, while a convex shape can support several users engaged on individual tasks. MistForm also enables unique interaction possibilities by exploiting the synergies between shape changing interfaces and mid-air fog displays. For instance, moving the screen will affect the brightness and blurriness of the screen at specific locations around the display, creating spaces with similar (collaboration) or different visibility (personalized content). We describe the design of MistForm and analyse its inherent challenges, such as image distortion and uneven brightness on dynamic curved surfaces. We provide a machine learning approach to characterize the shape of the screen and a rendering algorithm to remove aberrations. We finally explore novel interactive possibilities and reflect on their potential and limitations.

In the advent of a new age of natural user interaction with 3D gesture recognition cameras (e.g. Kinect, Leap Motion, Intel Real Sense, etc.), it is expected to display a floating image on top of a flat display to directly manipulate rendered objects and to utilize 3D interface with a full function. To present floating images with a conventional 2D display, researchers have studied diverse, passive re-imaging techniques to converge the displayed image in a midair. For instance, Repro3D [Yoshida et al. 2011] combined a pepper's ghost technique with a retro-reflective multi-array-projection to augment a floating, motion-parallax image with real scenes at the back of a half-mirror. Smoot et al. [Smoot et al. 2011] demonstrated a volumetric scanning mid-air display by a rim driven varifocal beam-splitter and a high-frame rate LED display. HaptoMIRAGE [Ueda et al. 2014] applied three large fresnel lens in a lateral direction to construct a wide-angle-view autostereoscopic display with a haptic feedback. Monnai [Monnai et al. 2014], Kim [Kim et al. 2014] and Maekawa [Maekawa et al. 2006] employed micro corner mirror arrays to form a clear real image with a wide range of focal points.