Litchi chinensis var. fulvosus is an important wild litchi resource in Yunnan, China, valued for favorable agronomic traits such as early flowering, early ripening, multiple flowering cycles, and high fruit-setting ability. However, its genetic diversity and population structure remain poorly understood. In this study, 192 accessions were collected from ten counties in Yunnan Province to evaluate their geographic distribution, leaf phenotypic variation, molecular diversity, population structure, and core collection composition. Eight descriptive leaf traits, nine quantitative leaf traits, and ISSR genotyping data from seven primers were analyzed. The accessions were distributed across an altitudinal range of 169–1470 m, with clear habitat differentiation among trees of different ages. Morphological analysis revealed substantial leaf variation, with mean diversity indices of 1.19 for descriptive traits and 2.76 for quantitative traits. ISSR analysis generated 49 scorable bands, of which 34 were polymorphic, corresponding to a polymorphism rate of 68.45%. The mean Shannon–Wiener diversity index was 0.3101, indicating detectable but relatively limited molecular diversity. Integrated phenotypic and molecular analyses divided the germplasm into two subpopulations. A core collection comprising 30 accessions (about 15% of the initial population) showed the best balance between sampling efficiency and diversity retention. These results provide a practical basis for the conservation, evaluation, and efficient utilization of L. chinensis var. fulvosus genetic resources and will support breeding and genetic improvement of litchi.