The automatic registration of infrared and visible images in vehicular imaging systems remains challenging in vision-assisted driving systems because of differences in imaging mechanisms. Existing registration methods often fail to accurately register infrared and visible images in vehicular imaging systems due to numerous spurious points during feature extraction, unstable feature descriptions, and low feature matching efficiency. To address these issues, a robust and efficient registration of infrared and visible images for vehicular imaging systems is proposed. In the feature extraction stage, we propose a structural similarity point extractor (SSPE) that extracts feature points using the structural similarity between weighted phase congruency (PC) maps and gradient magnitude (GM) maps. This approach effectively suppresses invalid feature points while ensuring the extraction of stable and reliable ones. In the feature description stage, we design a rotation-invariant feature descriptor (RIFD) that comprehensively describes the attributes of feature points, thereby enhancing their discriminative power. In the feature matching stage, we propose an effective coarse-to-fine matching strategy (EC2F) that improves the matching efficiency through nearest neighbor matching and threshold-based fast sample consensus (FSC), while improving registration accuracy through coordinate-based iterative optimization. Registration experiments on public datasets and a self-established dataset demonstrate the superior performance of our proposed method, and also confirm its effectiveness in real vehicular environments.