Recently, optical sensors interacting with evanescent fields and the external environment around waveguides have attracted extensive attention. In the process of light propagation in the waveguide, the depth of the evanescent field is closely related to the accuracy of the optical sensor, and adjusting the depth of the evanescent field to obtain higher accuracy has become the primary challenge in fabricating on-chip optical sensors. In this study, the waveguide structure of a Mach–Zehnder interferometer was written directly in Corning Eagle 2000 borosilicate glass by a femtosecond laser, and the sensing window was exposed out of the bulk material by mechanical polishing. The refractive index detection device based on the proposed on-chip Mach–Zehnder interferometer has the advantages of small volume, light weight, and good stability. Its sensitivity can reach 206 nm/RIU or 337 dB/RIU, and the theoretical maximum measurement range is 1–1.508. Therefore, it can measure the refractive index quickly and accurately in extreme or complex environments, and has excellent application prospects.