The fine-grained population distributions of different age groups are crucial for urban planning applications. With the development of information and communication technology (ICT), detailed population data retrieved from various big data sources, especially on a fine scale, have been extensively used for urban planning. However, studies estimating the detailed population distributions of different age groups are still lacking. This study constructs a framework to generate fine-grained population data for different age groups and explores the influence of various factors on the distributions of different age groups. The population is divided into the following four age groups: (1) early adulthood people: 18 ≤ age ≤ 24, (2) young people: 25 ≤ age ≤ 39, (3) middle-aged people: 40 ≤ age ≤ 59, and (4) elderly people: 60 ≤ age. The results indicate that education and accommodation factors have a major influence on the distributions of early adulthood and elderly people, respectively. Business, restaurant, and accommodation factors are the main factors influencing the population distributions of young and middle-aged people. The accommodation factor plays a major controlling role at night, and its explanatory power gradually decreases during the day, while the explanatory powers of the business and restaurant factors increase and become leading factors during the day. Specifically, the hospital factor has a greater effect on the distribution of elderly people. The entertainment factor has very little explanatory power for the population distributions of the different age groups.