Non-orthogonal multiple access (NOMA) techniques are widely used to increase quality-of-experience and network performance requirements in vehicle-to-everything (V2X) communication. However, dynamic vehicular communication conditions lead to the base station (BS) with limited knowledge about perfect channel state information (CSI), which incurs a challenging problem on spectrum resource allocation for complex communication systems with multiple transmission links. In particular, unstable communication of high-mobility vehicles degrades the performance of NOMA-V2X networks. To address these difficulties, this paper proposes a resource allocation scheme for vehicular communication to comprehensively consider user scheduling and power allocation, whereas it considers the channel fading in time-varying networks. By satisfying the constraints on the transmission power and rate of each user, the formulated problem aims to minimize the total system delay. To effectively solve the formulated problem, a deep deterministic policy gradient (DDPG) is deployed to find the solutions of the proposed scheme. Simulation results show that the proposed algorithm significantly outperforms the baseline in terms of both delay and convergence stability while satisfying realistic V2X constraints.