Although brain plasticity in the form of gray matter increases and decreases has been observed in chronic pain, factors determining the patterns of directionality are largely unknown. Here we tested the hypothesis that fibromyalgia interacts with age to produce distinct patterns of gray matter differences, specifically increases in younger and decreases in older patients, when compared to age-matched healthy controls. The relative contribution of pain duration was also investigated. Regional gray matter was measured in younger (n = 14, mean age 43, range 29-49) and older (n = 14; mean age 55, range 51-60) female fibromyalgia patients and matched controls using voxel-based morphometry and cortical thickness analysis of T1-weighted magnetic resonance images. To examine their functional significance, gray matter differences were compared with experimental pain sensitivity. Diffusion-tensor imaging was used to assess whether white matter changed in parallel with gray matter, and resting-state fMRI was acquired to examine whether pain-related gray matter changes are associated with altered functional connectivity. Older patients showed exclusively decreased gray matter, accompanied by compromised white matter integrity. In contrast, younger patients showed exclusively gray matter increases, namely in the basal ganglia and insula, which were independent of pain duration. Associated white matter changes in younger patients were compatible with gray matter hypertrophy. In both age groups, structural brain alterations were associated with experimental pain sensitivity, which was increased in older patients but normal in younger patients. Whereas more pronounced gray matter decreases in the posterior cingulate cortex were related to increased experimental pain sensitivity in older patients, insular gray matter increases in younger patients correlated with lower pain sensitivity, possibly indicating the recruitment of endogenous pain modulatory mechanisms. This is supported by the finding that the insula in younger patients showed functional decoupling from an important pain-processing region, the dorsal anterior cingulate cortex. These results suggest that brain structure and function shift from being adaptive in younger to being maladaptive in older patients, which might have important treatment implications.