Carbon dots (CDs) have aroused colossal attention in the fabrication of nanocomposite membranes ascribed to their ultra-small size, good dispersibility, biocompatibility, excellent fluorescence, facile synthesis, and ease of functionalization. Their unique properties could significantly improve membrane performance, including permeance, selectivity, and antifouling ability. In this review, we summarized the recent development of CDs-based nanocomposite membranes in many application areas. Specifically, we paid attention to the structural regulation and functionalization of CDs-based nanocomposite membranes by CDs. Thus, a detailed discussion about the relationship between the CDs’ properties and microstructures and the separation performance of the prepared membranes was presented, highlighting the advantages of CDs in designing high-performance separation membranes. In addition, the excellent optical and electric properties of CDs enable the nanocomposite membranes with multiple functions, which was also presented in this review.