This study proposes an adaptive compressed sensing (CS) method for video recovery based on spatial–temporal difference saliency in compressive domain. Firstly, the theoretical framework of realtime adaptive CS for video restoration is established, which includes two components: on‐chip and off‐chip. Secondly, the computation model of compressive domain spatial–temporal difference‐based saliency detection is built, which merely depends on the intra‐frame and inter‐frame similarities of compressive measurements. Thirdly, binary‐group approximation algorithm of subrate allocation is presented to achieve a quasi‐optimal solution. Finally, simulation experiments are designed to evaluate the performance of the proposed method, and it is indicated by the experimental results that the proposed method is suitable for realtime implementation and holds higher peak signal to noise ratio compared with the state‐of‐the‐art methods.