The loss of nitrogen and phosphate fertilizers in agricultural runoff is a global environmental problem, attracting worldwide attention. In the last decades, the constructed wetland has been increasingly used for mitigating the loss of nitrogen and phosphate from agricultural runoff, while the substrate, plants, and wetland structure design remain far from clearly understood. In this paper, the optimum substrates and plant species were identified by reviewing their treatment capacity from the related studies. Specifically, the top three suitable substrates are gravel, zeolite, and slag. In terms of the plant species, emergent plants are the most widely used in the constructed wetlands. Eleocharis dulcis, Typha orientalis, and Scirpus validus are the top three optimum emergent plant species. Submerged plants (Hydrilla verticillata, Ceratophyllum demersum, and Vallisneria natans), free-floating plants (Eichhornia crassipes and Lemna minor), and floating-leaved plants (Nymphaea tetragona and Trapa bispinosa) are also promoted. Moreover, the site selection methods for constructed wetland were put forward. Because the existing research results have not reached an agreement on the controversial issue, more studies are still needed to draw a clear conclusion of effective structure design of constructed wetlands. This review has provided some recommendations for substrate, plant species, and site selections for the constructed wetlands to reduce nutrients from agricultural runoff.