In this study, the accumulation characteristics of As, Hg, Cd, Cr, and Pb in 63 soil samples from 28 organic farms in Beijing, China, were analyzed to investigate the risk of heavy metal pollution in organic agriculture, and the key related factors were evaluated. The results revealed that the As, Hg, Cd, Cr, and Pb concentrations in the soil samples were below the risk screening values and substantially lower than those in the soil under conventional agriculture. However, the coefficients of variation for Hg and Cd were 112.45% and 38.34%, respectively, indicating a notable anthropogenic impact. Notably, 35.92% of the sampling sites had medium to high potential ecological risk values for Cd, and the Cd concentration increased considerably as the number of planting years increased. Different crop types impacted the soil heavy metal concentrations. The concentrations of Cd and As in the soil of Brassica crops were 0.265 and 12.915 mg/kg, respectively, which were substantially higher than those in the soil of other crop types. The Random Forest model indicated that soil nutrients had the most significant impact on soil heavy metal accumulation, particularly phosphorus. In conclusion, compared with conventional agriculture, organic agricultural soils have lower heavy metal concentrations and exhibit lower ecological risks, with no significant heavy metal pollution detected. However, there is a risk of Cd accumulation, and preventive measures should be implemented, especially for soils under prolonged cultivation and with potential sources of heavy Cd inputs.