Gradation segregation strongly affects the service performance of asphalt pavement. However, rare studies focus on the influences of segregation on asphalt mixture as it is difficult to simulate segregation in a laboratory. This paper presents a research aiming to evaluate the effects of gradation segregation using the two dimensional (2D) discrete element method (DEM). To achieve this objective, the gradation segregation levels are defined according to the differences between control and segregated gradations. Based on this, asphalt mixtures with three gradations at three segregation levels were selected. The corresponding 2D DEM models were then built using Particle Flow Code 2D (PFC 2D). The aggregate skeletons in the asphalt mixture were determined separately by means of the contact point and contact force. Based on this, the variations on the aggregate skeleton, aggregate distribution, and mechanical properties of the asphalt mixture under vertical load were studied at three different segregation levels. This study indicates that the aggregate skeleton in asphalt mixture can be affected by the gradation segregation. The correlation between gradation segregation and mechanical property is also determined in this study.