In this paper, the concept and definitions of the Fractional Order Moment (FOM) and Fractional Order Cumulant (FOC) are proposed, which is based on the fractional derivative of the fractional order Moment-generating function and the fractional order Cumulant-generating function of stochastic processes. The moment and cumulant are defined on an expanded set from positive integer to the whole positive real. This development not only provides a new technology for signal processing, also complements the existing theory in the field. The properties of the FOC have been derived, and their uniformity and particularity with the High Order Cumulant are compared and commented. In addition, the transformation between the FOM and the FOC are derived and discussed in detail. As one of the applications of the new concept to the α and Gaussian processes, a new method of suppressing α and Gaussian noise is proposed. Furthermore, a FOC-based parameter estimation algorithm is developed for the non-minimum phase ARMA processes in α and/or Gaussian noise. Simulation examples are used to demonstrate the effectiveness of the proposed parameter estimation algorithm.