Fano-resonant silicon metasurfaces exhibit extreme planar chirality, offering tremendous potential for miniaturized optical devices. However, achieving ultra-high quality factor (Q) resonance in such devices remains challenging. Here, we construct a fractal pentamer all-dielectric metasurface, of which the scale factors of second-order fractals are designed differently to introduce asymmetry. This asymmetry transforms symmetry-protected bound states in the continuum (BIC) into quasi-BIC (QBIC), achieving ultra-high Q factor Fano resonances.Magnetic dipole (MD) and transverse dipole (TD) BIC can be supported in this system, thus produce extremely narrow linewidth Fano resonances. By optimizing the asymmetric state, ultrahigh Q factor up to 4×104 is reached. We numerically obtain bulk sensitivities of 1.905 μm/RIU and figures of merit (FOM) up to 5625.5. The constructed resonances are insensitive to x and y polarizations due to the specific layout of clusters proposed here. Therefore, the proposed all-dielectric metasurface demonstrates good performance in refractive index sensing, which inspires the development of new high Q factor refractive index sensors for the nondestructive identification in the far-infrared regime.