The aim of this research was to select plant species that could be effective in the phytoremediation of a former oil-sludge pit. Seven crop plants (Triticum aestivum L., Secale cereale L., Avena sativa L., Hordeum vulgare, Sorghum bicolor L. Moench, Panicum miliaceum L., and Zea mays L.), five wild grasses (Lolium perenne L., Bromopsis inermis, Agropyron cristatum L., Agropyrum tenerum L., and Festuca pratensis Huds.), and three legumes (Medicago sativa L., Trifolium pratense L., and Onobrychis antasiatica Khin.) were screened for phytotoxicity, including the assessment of germination, shoot biomass, and root biomass, in a pot experiment. The estimation of oil-sludge degradation in the root zone of the tested plants showed that rye accelerated cleanup most effectively, degrading all of the main contaminant fractions in the oil sludge by a total of 52%. Although alfalfa had a lower phytoremediation potential than did rye, it maintained large numbers of soil microorganisms, including polycyclic aromatic hydrocarbon degraders, in its rhizosphere. Rye and alfalfa were chosen for a large-scale study to remediate an oil-sludge pit on the grounds of a petroleum refinery. Remediation monitoring confirmed the effectiveness of rye: the oil-sludge content decreased consistently for 3 years and remained low in comparison with the results from other plant species.