Smart grid has emerged as the next generation of power grid, due to its reliability, flexibility, and efficiency. However, smart grid faces some critical security challenges such as the message injection attack and the replay attack. If these challenges cannot be properly addressed, an adversary can maliciously launch the injected or replayed message attacks to degrade the performance of smart grid. To cope with these challenging issues, in this paper, we propose an efficient authentication scheme that employs the Merkle hash tree technique to secure smart gird communication. Specifically, the proposed authentication scheme considers the smart meters with computation-constrained resources and puts the minimum computation overhead on them. Detailed security analysis indicates its security strength, namely, resilience to the replay attack, the message injection attack, the message analysis attack, and the message modification attack. In addition, extensive performance evaluation demonstrates its efficiency in terms of computation complexity and communication overhead.