Filamentous fungi produce a diverse array of secondary metabolites (SMs) critical for defense, virulence, and communication. The metabolic pathways that produce SMs are found in contiguous gene clusters in fungal genomes, an atypical arrangement for metabolic pathways in other eukaryotes. Comparative studies of filamentous fungal species have shown that SM gene clusters are often either highly divergent or uniquely present in one or a handful of species, hampering efforts to determine the genetic basis and evolutionary drivers of SM gene cluster divergence. Here, we examined SM variation in 66 cosmopolitan strains of a single species, the opportunistic human pathogen Aspergillus fumigatus. Investigation of genome-wide within-species variation revealed 5 general types of variation in SM gene clusters: nonfunctional gene polymorphisms; gene gain and loss polymorphisms; whole cluster gain and loss polymorphisms; allelic polymorphisms, in which different alleles corresponded to distinct, nonhomologous clusters; and location polymorphisms, in which a cluster was found to differ in its genomic location across strains. These polymorphisms affect the function of representative A. fumigatus SM gene clusters, such as those involved in the production of gliotoxin, fumigaclavine, and helvolic acid as well as the function of clusters with undefined products. In addition to enabling the identification of polymorphisms, the detection of which requires extensive genome-wide synteny conservation (e.g., mobile gene clusters and nonhomologous cluster alleles), our approach also implicated multiple underlying genetic drivers, including point mutations, recombination, and genomic deletion and insertion events as well as horizontal gene transfer from distant fungi. Finally, most of the variants that we uncover within A. fumigatus have been previously hypothesized to contribute to SM gene cluster diversity across entire fungal classes and phyla. We suggest that the drivers of genetic diversity operating within a fungal species shown here are sufficient to explain SM cluster macroevolutionary patterns. All organisms produce metabolites, which are small molecules important for growth, reproduction, and other essential functions. Some organisms, including fungi, plants, and bacteria, make specialized forms of metabolites known as “secondary” metabolites that are ecologically important and improve their producers’ chances of survival and reproduction. In fungi, the genes in pathways that synthesize secondary metabolites are typically located next to each other in the genome and organized in contiguous gene clusters. These gene clusters, along with the metabolites they produce, are highly distinct, even between otherwise similar fungi, and it is often difficult to reconstruct how these differences evolved. To understand how secondary metabolic pathways evolve in fungi, we compared secondary metabolic gene clusters in 66 strains of one species of filamentous fungus, the human pathogen Aspergillus fumigatus. We show that these gene clusters vary extensively within this species, and describe the genetic processes that cause these differences. We identify 5 types of variants: single nucleotide changes, gene and gene cluster gain and loss, different gene clusters at the same genomic position, and mobile gene clusters that “jump” around the genome. These results provide a road map to the types and frequencies of genomic changes underlying the extensive diversity of fungal secondary metabolites.