In recent years, hyperspectral sparse unmixing (HSU) has garnered extensive research and attention due to its unique characteristic of not requiring the estimation of endmembers and their number. However, the high coherence and large-scale nature of the prior spectral library frequently lead to substantial computational costs and limited unmixing accuracy in the optimization model, thereby hindering the efficiency and further promotion of HSU in practical engineering applications. To address these shortcomings, this article proposes a new fast two-step sparse unmixing algorithm, called NeSU-LP, which is based on adaptive spectral library pruning technology and the Nesterov fast optimization strategy. In this method, HSU is divided into two independent and consecutive subprocesses: coarse unmixing and fine unmixing. Specially, first, in the coarse unmixing stage, we design a sparse optimization model based on the initial large spectral library, requiring only a few iterations to initially estimate the row-sparse abundance matrix. Subsequently, the proposed atomic (i.e., endmember) activity evaluation method is utilized to screen the active endmembers, analyze the abundance matrix, and prune the endmembers in the spectral library. Irrelevant endmembers are removed, reducing the spectral library size and generating a low-coherence, small-scale endmember matrix. Finally, in the fine unmixing process, we retain the effective atomic abundance rows obtained in the previous stage and design the final fine hyperspectral unmixing model based on the pruned, small-scale endmember matrix. In addition, to enhance the smoothness of the abundance maps, graph Laplacian regularization is introduced during the fine unmixing stage. The Nesterov fast gradient strategy is employed to accelerate the iterative process of fine unmixing, ultimately achieving second-order convergence efficiency for the algorithm. Numerous experiments were conducted on both synthetic and real datasets, comparing them with state-of-the-art methods. The experimental results demonstrate the high efficiency and advancement of the proposed method.