The Rhodiola species have a long history of utilization in traditional medicine and have been considered as a source of adaptation to environmental challenges; salidroside and p-tyrosol are the major responsible compounds. Here we propose a novel UPLC-guided two-step method consisting of a DIAION HP-20 adsorption and silica gel column chromatographies, which can simultaneously prepare high purities of salidroside and p-tyrosol with noticeable yields from the rhizome of Rhodiola crenulata. Results demonstrated that DIAION HP-20 could successfully remove all impurities except crenulatin during a gradient elution with 5–20% ethanol, which could achieve an optimal purification of salidroside and p-tyrosol with increasing rates of 29.19% and 33.44%, respectively. Furthermore, chloroform was selected as an ideal solvent for separating p-tyrosol with salidroside, and thus crenulatin was subsequently applied in the silica gel chromatography, and the separation of salidroside with crenulatin could be achieved using silica gel chromatography with a mixture of chloroform and methanol at a volume ratio of 4:1. High purity rates of 94.17% and 97.29% and overall yields of 39.09% and 43.73% for salidroside and p-tyrosol were simultaneously achieved. Our method provides a new way to simultaneously obtain salidroside and p-tyrosol from R. Crenulata, as well as other related plant species.