Plants have evolved interconnected regulatory pathways which enable them to respond and adapt to their environments. In plants, stress memory enhances stress tolerance through the molecular retention of prior stressful experiences, fostering rapid and robust responses to subsequent challenges. Mounting evidence suggests a close link between the formation of stress memories and effective future stress responses. However, the mechanism by which environmental stressors trigger stress memory formation is poorly understood. Here, we review the current state of knowledge regarding the RNA-based regulation on stress memory formation in plants and discuss research challenges and future directions. Specifically, we focus on the involvement of microRNAs (miRNAs), small interfering RNAs (siRNAs), long non-coding RNAs (lncRNAs), and alternative splicing (AS) in stress memory formation. miRNAs regulate target genes via post-transcriptional silencing, while siRNAs trigger stress memory formation through RNA-directed DNA methylation (RdDM). lncRNAs guide protein complexes for epigenetic regulation, and AS of pre-mRNAs is crucial to plant stress memory. Unraveling the mechanisms underpinning RNA-mediated stress memory formation not only advances our knowledge of plant biology but also aids in the development of improved stress tolerance in crops, enhancing crop performance and global food security.