We report on the temperature dependence of selective phonon scattering and magnetization switching in perovskites. Thermal evolution of key phonon modes and their anomalous change across the antiferromagnetic (AFM) Néel temperature endorses the strong spin-phonon coupling in the investigated system. Such changes have been identified in consonance with the vibrational spectrum of trivalent Gd and Cr along with antistretching/bending of polyhedra and octahedra. Further, we find the emergence of many interesting features, such as thermomagnetic irreversibilities of the magnetization, significant enhancement of the ordering temperatures (up to 202 K), and multiple magnetization switching below (9.7–85 K) upon the substitution of Sm at Gd sites. We find the presence of kinetic arrest of magnetic phases and (; )] along with the presence of nonequilibrium frozen AFM and ferromagnetic clusters ( K with 70.7% frozen fraction), indicating the presence of a magnetic glassy state which assists the phase transition and supports the phase-coexistence nature of the perovskites. Interestingly, Sm moments order across 6.82 K (), giving rise to yet another phase transition: . On the other hand, field-dependent bipolar switching of magnetization at different temperatures below under different measurement protocols makes the investigated system a suitable candidate for the magnetic memories and magnetic switches.