Output probability density function (PDF) tracking control of stochastic systems has always been a challenging problem in both theoretical development and engineering practice. Focused on this challenge, this work proposes a novel stochastic control framework so that the output PDF can track a given time-varying PDF. Firstly, the output PDF is characterised by the weight dynamics following the B-spline model approximation. As a result, the PDF tracking problem is transferred to a state tracking problem for weight dynamics. In addition, the model error of the weight dynamics is described by the multiplicative noises to more effectively establish its stochastic dynamics. Moreover, to better reflect the practical applications in the real world, the given tracking target is set to be time-varying rather than static. Thus, an extended fully probabilistic design (FPD) is developed based on the conventional FPD to handle multiplicative noises and to track the time-varying references in a superior way. Finally, the proposed control framework is verified by a numerical example, and a comparison simulation with the linear–quadratic regulator (LQR) method is also included to illustrate the superiority of our proposed framework.