In this study, the cytotoxicity and toxic mechanism of carbon quantum dots (CQDs) to E. coli were evaluated in vitro. The synthetic CQDs were extremely small in size (~2.08 nm) and displayed strong fluorescence. The results demonstrated that CQDs showed good biocompatibility with E. coli within a short culture time. However, when the exposure time exceeded 24 h, the toxicity of CQDs became apparent, and the contents of reactive oxygen species, lactate dehydrogenase, and the crystal violet absorption rate increased significantly. To further explore the cytotoxic mechanism, approaches including confocal laser scanning microscopy, scanning electron microscopy, and biological transmission electron microscopy combined with zeta potential tests, osmotic pressure measurement, and comet assays were performed. On the one hand, the CQDs altered the surface charges of cells and induced lipid peroxidation by adhesion on the surface of E. coli, leading to an increase in the permeability of the cell wall. On the other hand, when the concentration of CQDs reached 200 µg/mL, the osmotic pressure of the extracellular environment was significantly reduced. These are the main factors that lead to cell edema and death. Finally, the comet assays confirmed that CQDs could induce DNA damage, which could inhibit the proliferation of E. coli.