The development of the concrete industry is always accompanied by some environmental issues such as global warming and energy consumption. Under this circumstance, the application of nanocellulose in cementitious materials is attracting more and more attention in recent years not only because of its renewability and sustainability but also because of its unique properties. To trace the research progress and provide some guidance for future research, the application of nanocellulose to cementitious materials is reviewed. Specifically, the effects of cellulose nanocrystal (CNC), cellulose nanofibril (CNF), bacterial cellulose (BC), and cellulose filament (CF) on the physical and fresh properties, hydration, mechanical properties, microstructure, rheology, shrinkage, and durability of cementitious materials are summarized. It can be seen that the type, dosage, and dispersion of nanocellulose, and even the cementitious matrix type can lead to different results. Moreover, in this review, some unexplored topics are highlighted and remain to be further studied. Lastly, the major challenge of nanocellulose dispersion, related to the effectiveness of nanocellulose in cementitious materials, is examined in detail.