In this article, we investigate the approximation ability of recurrent neural networks (RNNs) with stochastic inputs in state space model form. More explicitly, we prove that open dynamical systems with stochastic inputs can be well-approximated by a special class of RNNs under some natural assumptions, and the asymptotic approximation error has also been delicately analyzed as time goes to infinity. In addition, as an important application of this result, we construct an RNN-based filter and prove that it can well-approximate finite dimensional filters which include Kalman filter (KF) and Beneš filter as special cases. The efficiency of RNN-based filter has also been verified by two numerical experiments compared with optimal KF.

Direct method for Yau filtering system has been studied since 1990s and all these work are limited in time-invariant systems. In this work, we extend the direct method so that it is applicable to time-varying cases. We need less assumptions compared with our previous work. The novelty of this work is that we propose several transformations on the forward Kolmogorov equation so that it can be solved by means of solving some ordinary differential equations if the initial distribution is Gaussian. The corresponding results for any non-Gaussian initial distributions can be obtained via Gaussian approximation. It can be seen that with our new scheme direct method can treat nearly most general Yau filtering problem under natural assumptions. Our algorithm has been compared with the extended Kalman filter, multilevel particle filter and ensemble Kalman filter by numerical examples and the simulation results show the efficiency of our method.

Ever since the technique of Kalman filter was popularized, there has been an intense interest in finding new classes of finite dimensional recursive filters. In the late seventies, the idea of using estimation algebra to construct finite-dimensional nonlinear filters was first proposed by Brockett and Mitter independently. It has been proven to be an invaluable tool in the study of nonlinear filtering problem. For all known finite dimensional estimation algebras, the Wong's Ω-matrix has been proven to be a constant matrix. However, the Wong's Ω-matrix is shown not necessary to be a constant matrix in this letter when we consider finite dimensional estimation algebras with state dimension 3 and rank equal to 1. Several easily satisfied conditions are established for an estimation algebra of a special class of filtering systems to be finite-dimensional. Finally, we give the construction of finite dimensional filters of non-maximal rank.

In this paper, we derive an optimal transportation particle filter for linear time-varying systems with correlated noises. This method can be regarded as the extension of the feedback particle filter with optimal transportation structure. However, the particles in our method are evolved in a deterministic way while we need to generate random particles in feedback particle filter. Consequently, we only need a very few particles to obtain the satisfying results and this property is especially significant for high dimensional problems. The error analysis of our method and feedback particle filter have been carried out when the system is time-invariant. Compared with feedback particle filter and ensemble Kalman filter, our method shows great efficiency in numerical experiments including both scalar case and high dimensional case.

In numerous application areas, high-dimensional nonlinear filtering is still a challenging problem. The introduction of deep learning and neural networks has improved the efficiency of classical algorithms and they perform well in many practical tasks. However, a theoretical interpretation of their feasibility is still lacking. In this paper, we exploit the representational ability of recurrent neural networks and provide a computationally efficient and optimal framework for nonlinear filter design based on the Yau-Yau algorithm and recurrent neural networks. Theoretically, it can be proved that the size of the neural network required in this algorithm increases only polynomially rather than exponentially with dimension, which implies that the Yau-Yau algorithm based on recurrent neural networks has the ability to overcome the curse of dimensionality. Numerical results also show that our method is more competitive than classical algorithms for high-dimensional problems.