Generalized Labeled Multi-Bernoulli (GLMB) densities arise in a host of multi-object system applications analogous to Gaussians in single-object filtering. However, computing the GLMB filtering density requires solving NP-hard problems. To alleviate this computational bottleneck, we develop a linear complexity Gibbs sampling framework for GLMB density computation. Specifically, we propose a tempered Gibbs sampler that exploits the structure of the GLMB filtering density to achieve an $\mathcal {O}(T(P+M))$ complexity, where $T$ is the number of iterations of the algorithm, $P$ and $M$ are the number hypothesized objects and measurements. This innovation enables the GLMB filter implementation to be reduced from an $\mathcal {O}(TP^{2}M)$ complexity to $\mathcal {O}(T(P+M+\log T)+PM)$ . Moreover, the proposed framework provides the flexibility for trade-offs between tracking performance and computational load. Convergence of the proposed Gibbs sampler is established, and numerical studies are presented to validate the proposed GLMB filter implementation.

This article presents the Labeled Random Finite Set (LRFS) framework for multi-object systems-systems in which the number of objects and their states are unknown and vary randomly with time. In particular, we focus on state and trajectory estimation via a multi-object State Space Model (SSM) that admits principled tractable multi-object tracking filters/smoothers. Unlike the single-object counterpart, a time sequence of states does not necessarily represent the trajectory of a multi-object system. The LRFS formulation enables a time sequence of multi-object states to represent the multiobject trajectory that accommodates trajectory crossings and fragmentations. We present the basics of LRFS, covering a suite of commonly used models and mathematical apparatus (including the latest results not published elsewhere). Building on this, we outline the fundamentals of multi-object state space modeling and estimation using LRFS, which formally address object identities/trajectories, ancestries for spawning objects, and characterization of the uncertainty on the ensemble of objects (and their trajectories). Numerical solutions to multiobject SSM problems are inherently far more challenging than those in standard SSM. To bridge the gap between theory and practice, we discuss state-of-the-art implementations that address key computational bottlenecks in the number of objects, measurements, sensors, and scans.

If computational tractability were not an issue, multi-object estimation should integrate all measurements from multiple sensors across multiple scans. In this article, we propose an efficient numerical solution to the multi-scan multi-sensor multi-object estimation problem by computing the (labeled) multi-sensor multi-object posterior density. Minimizing the $L_{1}$ -norm error from the exact posterior density requires solving large-scale multi-dimensional assignment problems that are NP-hard. An efficient multi-dimensional assignment algorithm is developed based on Gibbs sampling, together with convergence analysis. The resulting multi-scan multi-sensor multi-object estimation algorithm can be applied either offline in one batch or recursively. The efficacy of the algorithm is demonstrated using numerical experiments with a simulated dataset.

Meeting or conference assistance is a popular application that typically requires compact configurations of co-located audio and visual sensors. This paper proposes a novel solution for online separation of an unknown and time-varying number of moving sources using only a single microphone array co-located with a single visual device. The approach exploits the complementary nature of simultaneous audio and visual measurements, accomplished by a model-centric 3-stage process of detection, tracking, and (spatial) filtering, which performs separation in a block-wise or recursive fashion. Fusing the measurements requires solving the multi-modal space-time permutation problem, since the audio and visual measurements reside in different observation spaces, but also are unidentified or unlabeled (with respect to the unknown and time-varying number of sources), and are subject to noise, extraneous measurements and missing measurements. A labeled random finite set tracking filter is applied to resolve the permutation problem and recursively estimate the source identities and trajectories. A time-varying set of generalized side-lobe cancellers is constructed based on the tracking estimates to perform online separation. Evaluations are undertaken with live human speakers.