The Taiyuan Formation, located on the northern margin of the Ordos Basin, has an important role in reconstructing tectonic paleogeography and evolutionary process due to its preservation of information about the tectonic evolution of the basin throughout the Late Paleozoic. The provenance, paleogeography, and tectonic activity traits of the Taiyuan Formation were examined using the sedimentary petrology and zircon chronology methods. The Alxa block is consistent with the dating results of the Taiyuan Formation in Shabatai, and the Daqing Shan is consistent with the results of the Taiyuan Formation in Xiachengwan and Adaohai. It is inferred that the provenance of the Taiyuan Formation in the western part of the northern margin of the basin is from the Alxa block, and the provenance in the eastern part is from the Daqing Shan when combined with the study of sandstone detrital components, and paleocurrent. In the late Paleozoic, the Siberian plate collided with the North China Craton, and the Taiyuan Formation was deposited in the northern margin of the Ordos Basin. However, on account of the different ages and types of basement rocks and the central paleo-uplift which has not completely disappeared, the Taiyuan Formation has obvious east-west zonation.