Projection mapping (PM) exhibits suboptimal performance in well-lit environments because of the interference caused by ambient light. This interference degrades the contrast of the projected images. Consequently, conventional methodologies restrict the application of PM to dimly lit settings, leading to an unnatural visual experience, as only the PM target is prominently illuminated. To overcome these limitations, we introduce an innovative approach that leverages a mixed light field, blending traditional PM with ray-controllable ambient lighting. This methodological combination, despite its simplicity, is effective because it ensures that the projector exclusively illuminates the PM target, preserving the optimal contrast. Precise control of ambient light rays is essential to prevent them from illuminating the PM target while adequately illuminating the surrounding environment. Furthermore, we propose the integration of a kaleidoscopic array with integral photography to generate dense light fields for ray-controllable ambient lighting. Additionally, we present an efficient binary-search-based calibration method tailored to this intricate optical system. Our optical simulations and the developed system collectively validate the effectiveness of our approach. Our results show that PM targets and ordinary objects coexist naturally in environments that are brightly lit as a result of our method, enhancing the overall visual experience.
Dynamic projection mapping for a moving object according to its position and shape is fundamental for augmented reality to resemble changes on a target surface. For instance, augmenting the human arm surface via dynamic projection mapping can enhance applications in fashion, user interfaces, prototyping, education, medical assistance, and other fields. For such applications, however, conventional methods neglect skin deformation and have a high latency between motion and projection, causing noticeable misalignment between the target arm surface and projected images. These problems degrade the user experience and limit the development of more applications. We propose a system for high-speed dynamic projection mapping onto a rapidly moving human arm with realistic skin deformation. With the developed system, the user does not perceive any misalignment between the arm surface and projected images. First, we combine a state-of-the-art parametric deformable surface model with efficient regression-based accuracy compensation to represent skin deformation. Through compensation, we modify the texture coordinates to achieve fast and accurate image generation for projection mapping based on joint tracking. Second, we develop a high-speed system that provides a latency between motion and projection below 10 ms, which is generally imperceptible by human vision. Compared with conventional methods, the proposed system provides more realistic experiences and increases the applicability of dynamic projection mapping.
This paper focuses on development of a high-speed, low-latency telemanipulated robot hand system, evaluation of the system, and demonstration of the system. The characteristics of the developed system are the followings: non-contact, high-speed 3D visual sensing of the human hand, intuitive motion mapping between human hands and robot hands, and low-latency, fast responsiveness to human hand motion. Such a high-speed, low-latency telemanipulated robot hand system can be considered to be more effective from the viewpoint of usability. The developed system consists of a high-speed vision system, a high-speed robot hand, and a real-time controller. For the developed system, we propose new methods of 3D sensing, mapping between the human hand and the robot hand, and the robot hand control. We evaluated the performance (latency and responsiveness) of the developed system. As a result, the latency of the developed system is so small that humans cannot recognize the latency. In addition, we conducted experiments of opening/closing motion, object grasping, and moving object grasping as demonstrations. Finally, we confirmed the validity and effectiveness of the developed system and proposed method.
The existing phase-shift methods are effective in achieving high-speed, high-precision, high-resolution, real-time shape measurement of moving objects; however, a phase-unwrapping method that can handle the motion of target objects in a real environment and is robust against global illumination as well is yet to be established. Accordingly, a robust and highly accurate method for determining the absolute phase, using a minimum of three steps, is proposed in this study. In this proposed method, an order structure that rearranges the projection pattern for each period of the sine wave is introduced, so that solving the phase unwrapping problem comes down to calculating the pattern order. Using simulation experiments, it has been confirmed that the proposed method can be used in high-speed, high-precision, high-resolution, three-dimensional shape measurements even in situations with high-speed moving objects and presence of global illumination. In this study, an experimental measurement system was configured with a high-speed camera and projector, and real-time measurements were performed with a processing time of 1.05 ms and a throughput of 500 fps.
The recent developments in 3D display technology are remarkable. Owing to such developments, the importance of methods for interacting with 3D images has been increasing. In particular, the ability to perform input operations by directly touching displayed images is important. Therefore, a sensing method is required for recognizing the 3D positions of fingers and determining whether the fingers are touching the 3D image displayed. Conventionally, such a sensing method generally involves position sensing of fingers through image-based active sensing. However, this does not solve the following two problems: (1) the problem of positional registration, that is, the accurate matching between the displayed image and input location, and (2) the problem of occlusion robustness, that is, the achievement of successful sensing even with the presence of an occluding object between the sensor and hands to allow free movement of the hands. Our proposed method solves these problems through the following two ideas. First, we used a method called aerial imaging by retroreflection, which focuses light rays from a wider range than other 3D display systems. Second, for capturing the reflected light as well as projecting to the sensing target in active sensing, we used the light-field formed by the 3D display system. We also propose a method for obtaining rotation information, considering that the light-field formed by the reflected light changes based on the angle of the sensing target. Simulations and experiments were performed to evaluate the proposed system, and the system setup was optimized through simulations. In the first two experiments conducted on position and rotation sensing, we evaluated the sensing errors due to occlusion and found them to be less than 1.74 mm and less than 15.4 deg, respectively. In the third experiment, we constructed an interaction system with 3D images by using the proposed method and evaluated this system.
We present a novel calibration method for a multi-view laser Doppler speed sensing (MLDSS) system. In contrast with the traditional method where only the laser geometry is independently calibrated, the proposed method simultaneously optimizes all the laser parameters and directly associates the parameters with a motion sensing model. By jointly considering the consistency among laser Doppler velocimetry, the laser geometry and a visual marker tracking system, the proposed calibration method further boosts the accuracy of MLDSS. We analyzed the factors influencing the precision, and quantitatively evaluated the efficiency of the proposed method on several data sets.