Since China’s reform and opening up in 1978, the reclamation and abandonment of cropland in Xinjiang have become significant features of the land use change in the arid land of Northwest China. However, the spatiotemporal changes and driving mechanisms of cropland reclamation and abandonment over long time periods are still unclear, but this is crucial in understanding cropland changes in inland arid land, providing important insights for land management and agricultural development. Based on 40 years of remote sensing data on resources and the environment, this study examines the spatiotemporal characteristics of cropland reclamation and abandonment in Xinjiang over four periods since 1980. Additionally, it uses an optimal parameter geographical detector model to quantify the driving factors for each period. The results indicate that cropland reclamation experiences a “slow decrease–rapid increase” trend, forming a “V-shaped” pattern, while abandonment shows a “rapid decrease–slow decrease–slow increase” trend, forming a “U-shaped” pattern. These trends can be divided into three periods: 1980–1990 (unstable growth), 1990–2010 (stable growth), and 2010–2020 (growth with constraints). The movement pattern of cropland reclamation’s center of gravity is “slightly southeast–slightly northeast–southwest”, whereas the abandonment’s center of gravity shifts “northeast–southwest–northeast”. Further analysis reveals that the impact of agricultural technological investment and infrastructure on cropland reclamation has increased, while the influence of natural environmental factors has decreased. Although climate and water resources remain key factors in cropland abandonment, the influence of economic and social factors has gradually diminished, and the impact of agricultural mechanization has steadily risen.