For the application of proton exchange membrane fuel cells (PEMFCs) in a wide temperature range, an ionogel incorporated into poly(2,5-benzimidazole) (ABPBI) membranes with low phosphoric acid doping levels was designed. The natural sepiolite was first acid-treated to obtain one-dimensional silicon nanorods (SNRs) with a large specific surface area and a hierarchical porous structure. The SNRs were then filled with imidazolium ionic liquid (IL) to prepare IL@SNR ionogels. Analysis of related spectra and thermal behavior verified that the IL molecules were confined in the inner channels of SNRs. The as-prepared IL@SNRs were embedded in the ABPBI to form composite membranes (ABPBI/IL@SNRs) via in situ synthesis. By immobilizing proton conductors, the ABPBI/IL@SNR composite membranes markedly improved proton conductivity with low phosphoric acid doping levels in a wide temperature range. A single-cell based on 5 wt% IL@SNRs embedded in a composite membrane achieved a maximum power density of 0.15 and 0.28 W cm⁻² at 80 °C and 180 °C, respectively, with 0% RH. The proton conductivities were comparable to those of Nafion-based PEMFCs under the same temperature and humidity conditions and competed with those of reported polybenzimidazole–inorganic composite membrane-based PEMFCs at high temperatures. By using a facile method to prepare nanostructured fillers, the strategy of avoiding proton carrier wastage can be potentially used in the production of high-performance membranes for PEMFCs.