In this paper, we present a noise-robust approach for the 3D pose estimation of multiple people using appearance similarity. The common methods identify the cross-view correspondences between the detected keypoints and determine their association with a specific person by measuring the distances between the epipolar lines and the joint locations of the 2D keypoints across all the views. Although existing methods achieve remarkable accuracy, they are still sensitive to camera calibration, making them unsuitable for noisy environments where any of the cameras slightly change angle or position. To address these limitations and fix camera calibration error in real-time, we propose a framework for 3D pose estimation which uses appearance similarity. In the proposed framework, we detect the 2D keypoints and extract the appearance feature and transfer it to the central server. The central server uses geometrical affinity and appearance similarity to match the detected 2D human poses to each person. Then, it compares these two groups to identify calibration errors. If a camera with the wrong calibration is identified, the central server fixes the calibration error, ensuring accuracy in the 3D reconstruction of skeletons. In the experimental environment, we verified that the proposed algorithm is robust against false geometrical errors. It achieves around 11.5% and 8% improvement in the accuracy of 3D pose estimation on the Campus and Shelf datasets, respectively.

3D pose estimation of multi-person in real-time is a challenging problem which is essential for a various virtual reality (VR) applications. In this paper, we propose a real-time multi-person 3D pose estimation system for live streaming of 3D animation with multiple RGB-D cameras. The proposed system comprises the several edge devices connected to a central server via networks. 2D pose detection and depth sensing is locally conducted on each edge device. The edge device transmits the results to the central server. 3D pose reconstruction is performed on the central server. The central server aligns the multiple camera coordinates with a specific world plane. Then, the central server matches the 2D pose result across the multiple cameras to a person based on the distance. Finally, the 3D poses are reconstructed based on multi-view triangulation. The proposed system is capable of processing in real-time. To demonstrate the proposed system estimates multi-person 3D poses in real-time, we implement the prototypes and show the proposed system can be applied to the live streaming of 3D animation such as PC and Web.

KSII Transactions on Internet and Information Systems TIIS Paper Details

With the growing interest in digital twin technology, there has been a rise in efforts to apply technology in diverse fields. The digital twin can provide opportunities for humans to benefit in both industrial and daily life. We propose a digital twin platform, MirrorLake, designed to support the effective construction and utilization of digital twins by providing core functionalities aligned with the unique characteristics of digital twins. We proposed the digital twin resource management system to handle the representative components of the digital twin that emerged from our in-depth research analysis. We also implemented a synchronization engine and its ecosystem that supports high-volume and bidirectional synchronization between the real and virtual worlds. The engine also supports synchronization between the components of the digital twin by accommodating heterogeneity and linking real-time data. We demonstrated the utility of MirrorLake through three real-world applications in the city environment, smart office, and public safety, highlighting its potential to facilitate digital transformation in various industries.

Crowd-related disasters in dense public spaces unfold into hazardous situations within seconds, repeatedly demonstrating that reactive response alone is insufficient to minimize damage. This reality has intensified the need for monitoring systems that can proactively forecast congestion before it reaches a critical level. Digital twin platforms address this need by providing an operational substrate that represents crowd states on a unified bird’s-eye-view (BEV) grid, on which a predictive module can forecast where congestion will emerge. However, conventional AI-based single-stage dense prediction models are intrinsically ill-suited to this role: although crowd congestion is sparse in both space and time, these models apply uniform high-resolution computation across the entire BEV domain, wasting computation and biasing optimization toward dominant background regions. In this paper, we propose a two-stage coarse-to-fine framework that operates as the predictive module of the digital twin and explicitly exploits the spatio-temporal sparsity of crowd congestion. The first stage, CoarseSTFormer, performs efficient global screening on a low-resolution BEV input to coarsely identify a set of density-critical candidate regions. The second stage, SparseQueryDecoder, selectively reconstructs high-resolution responses only on the identified candidates, rather than uniformly upsampling the entire BEV grid. In simulation environments with up to 20,000 pedestrian agents, the proposed framework matches the strongest dense baseline in reconstruction quality while delivering the most balanced variance profile across grid scales. At inference, it further reduces GPU energy consumption by $1.9\times$ to $5.0\times$ and computational cost (FLOPs) by $3.8\times$ to $54\times$, demonstrating its practicality as a resource-efficient predictive module that satisfies both accuracy and efficiency.