Nowadays, with the advancement of the Internet and personal mobile devices, many interactive media are prevailing, where viewers make their own decisions on the story of the media based on their interactions. The interaction that the user can make is usually pre-programmed by a programmer. Therefore, interactions that users can make are limited to programmable areas. In comparison, in this paper, we propose an Interactive media authoring tool which can compose diverse two-hand interactions from several one-hand interactive components. The aim is to provide content creators with a tool to produce multiple hand motions so that they can design a variety of user interactions to stimulate the interest of content viewers and increase their sense of immersion. Using the proposed system, the content creator can gain greater freedom to create more diverse and complex interactions than programmable ones. The system is composed of a complex motion editor that edits one-hand motions into complex two-hand motions, a touchless sensor that senses the hand motion and a metadata manager that handles the metadata, which specify the settings for the interactive functions. To our knowledge, the proposed system is the first web-based authoring tool that can authorize complex two-hand motions from single hand motions, and which can also control a touchless motion control device.

Accurate alignment of real-world object poses with their virtual counterparts using sensors, e.g. cameras, is essential for consistent interaction in mixed-reality systems. However, objects can undergo abrupt, untracked movements during periods when a tracking system is inactive, e.g., overnight, causing stored pose records to become inconsistent with the real scene and breaking user interaction in the virtual environment. Off-the-shelf 3D reconstruction networks such as MASt3R (Matching and Stereo 3D Reconstruction) method provide metrically scaled 3D point maps and pixel correspondences, but they are trained on static scenes and therefore fail to produce reliable object correspondences when the object has moved. We propose a robust pipeline that combines MASt3R’s metrically scaled 3D outputs with a background-based alignment strategy to recover and apply the true pose change of moved objects. Our method first segments foreground and background and extracts 3D background point sets for a reference day and a current day. An affine transformation between these background point sets is estimated via a standard registration technique and used to express the current-day object 3D coordinates in the reference coordinate frame. Within that unified frame we compute the object pose change and apply the resulting transform to the virtual object, restoring real–virtual consistency. Experiments on real scenes demonstrate that the proposed approach reliably corrects pose misalignments introduced during inactive periods and substantially improves over applying MASt3R alone, thereby enabling restored and consistent user interaction in the virtual environment.

As the demand for immersive content that facilitates physical interactions with users continues to rise, so does the need for such content. The evolution of web technologies is expected to lead to a surge in 3D content that can be executed directly within web environments, providing a 3D experience across multiple platforms. The increasing availability of 3D content creation tools is likely to empower both professionals and novices to produce content more easily. Given the growing demand for 3D content, the trend towards cross-platform availability, and the simplification of content authoring tools, technologies that enable the production of interactive 3D content through Multi-Device Integration in web contexts are set to become essential. To address this demand, this paper presents an interaction API designed to streamline user interactions within web environments. By utilizing this API, developers can create interactive 3D content across various devices. The Interactive 3D Media Production framework enhances usability, accessibility, and platform independence by integrating diverse interaction modalities and providing plug-ins for other authoring tools.