It is difficult to simultaneously achieve surface-enhanced Raman scattering (SERS) and surface-enhanced fluorescence (SEF) for noble metals. Herein, a composite substrate is demonstrated based on the rational construction of Ag nanoparticles (Ag NPs) and inverse opal polydimethylsiloxane (PDMS) for surface Raman fluorescence dual enhancement. The well-designed Ag nanoparticle (Ag NP)-decorated inverse opal PDMS (AIOP) composite substrate is fabricated using the polystyrene (PS) photonic crystal method and the sensitization reduction technique. The inverse opal PDMS enhances the electromagnetic (EM) field by increasing the loading of Ag NPs and plasmonic coupling of Ag NPs, leading to SERS activity. The thin shell layer of polyvinyl pyrrolidone (PVP) in core–shell Ag NPs isolates the detected molecule from the Ag core to prevent the fluorescence resonance energy transfer and charge transfer to eliminate fluorescence quenching and enable SEF performance. Based on the blockage of the core–shell structure and the enhanced EM field originating from the inverse opal structure, the as-fabricated AIOP composite substrate shows dual enhancement in surface Raman fluorescence. The AIOP composite substrate in this work, which combines improved SERS activity and SEF performance, not only promotes the development of surface-enhanced spectroscopy but also shows promise for applications in flexible sensors.