Tunable and low-power microcavities are essential for large-scale photonic integrated circuits. Thermal tuning, a convenient and stable tuning method, has been widely adopted in optical neural networks and quantum information processing. Recently, graphene thermal tuning has been demonstrated to be a power-efficient technique, as it does not require thick spacers to prevent light absorption. In this paper, a silicon-based on-chip Fano resonator with graphene nanoheaters is proposed and fabricated. This novel Fano structure is achieved by introducing a scattering block, and it can be easily fabricated in large quantities. Experimental results demonstrate that the resonator has the characteristics of a high quality factor (∼31,000) and low state-switching power (∼1 mW). The temporal responses of the microcavity exhibit qualified modulation speed with 9.8 μs rise time and 16.6 μs fall time. The thermal imaging and Raman spectroscopy of graphene at different biases were also measured to intuitively show that the tuning is derived from the joule heating effect of graphene. This work provides an alternative for future large-scale tunable and low-power-consumption optical networks, and has potential applications in optical filters and switches.