The study of the spatial-temporal pattern and social performance of urban public services is a basic task for achieving urban fairness and justice. Through spatial analysis and social performance evaluation, this study explores the evolution of spatial-temporal patterns and the social performance of community sports and fitness venues in Shanghai from 1982 to 2019. The results show that the construction of Shanghai’s community sports and fitness venues presents the evolution pattern of “urbanization-suburbanization-reurbanization”. The center of construction has always been in the urban area and first moved toward the south and then toward the north. Government investment was the main source of funds for the construction of venues, and social investment has been steadily growing. The number and area of multiple types of venues has increased significantly, including trails, gymnasiums, and courts. The overall service coverage radius of Shanghai’s community sports and fitness venues has been significantly increased, and the regional equality between the core and peripheral areas has been obviously improved. The overall per capita service location entropy has not been significantly improved. The old city center and the peripheral area have always been the low-value areas, and the old city center is surrounded by high-value areas. The “low-high-low” three-circle spatial structure continues to exist, but around the old city center, the scope of the high-value area has expanded markedly. There was a significant optimization of social performance from 1999 to 2009. The social performance of the community sports and fitness venues in urban areas is better than that in suburban areas, but the optimization of social performance in suburban areas is greater than that in urban areas. The above analysis is expected to provide references for rationally arranging urban sports and fitness spaces, enhancing the fairness of urban public services, improving the quality of residents’ lives, and assisting the implementation of the “Healthy China” national strategy.