Flammulina filiformis (F. filiformis) is called the ‘benefiting intelligence’ mushroom. There is a notable difference between a yellow cultivar (with a robust aroma) and a white mutant cultivar (with a high yield) of F. filiformis. A thorough analysis of aroma differences is essential to improve the aroma of high-yield strains. This study employed a combination of gas chromatography–mass spectrometry–olfactometry (GC-MS-O) and aroma extract dilution analysis (AEDA) to analyze the variations in aroma compounds. Then, the contribution of the odorants was determined using flavor dilution (FD) factors and odor activity values (OAVs). Aroma omission and recombination experiments were used to identify the key odorants. A total of 16 key aroma compounds were characterized in F. filiformis, along with four eight-carbon volatiles (3-octanone, 3-octanol, octanal, and 1-octen-3-ol). Finally, the dominant aroma characteristic was “sweet” for the yellow strain, while it was “green” for the white strain. More research is required to investigate the enzymes and corresponding genes that regulate the synthesis of aroma compounds in F. filiformis for future breeding programs.