Rapid and reliable damaged building identification after earthquakes is of great significance for emergency rescue and post-disaster assessment. Remote sensing imagery provides important technical support for post-earthquake building damage assessment. To address the strong interference from background land-cover types in earthquake-stricken areas and the insufficient stability of damage identification based on single data source, this study proposes an optical–SAR multi-feature collaborative framework for damaged building identification. Firstly, by using pre-earthquake GF-2 optical imagery and LuTan-1 SAR imagery, a random forest classifier is employed using texture and morphological features to obtain the initial pixel-level building classification result, which is then refined through object-level post-processing to obtain the final building-area extraction result. Subsequently, two dual-polarization normalized change features based on pre- and post-earthquake Sentinel-1 SAR imagery are proposed to characterize variations in building scattering intensity, while the interferometric coherence feature derived through D-InSAR processing is used to measure structural changes. Finally, by fusing the above three features, a strategy combining thresholding and majority voting is adopted to identify damaged buildings. Experimental results indicate that the proposed method shows promising performance. First, for pre-earthquake building-area extraction, the proposed object-based method achieved an mIoU of 86.46%. Second, compared with the selected comparison methods, the proposed identification method achieved an overall accuracy of 81.75%. The proposed method can suppress non-building land-cover interference and enhance the stability and reliability of damaged building identification in complex areas.