Healthy aging (HA) is associated with certain declines in cognitive functions, even in individuals that are free of any process of degenerative illness. Functional magnetic resonance imaging (fMRI) has been widely used in order to link this age-related cognitive decline with patterns of altered brain function. A consistent finding in the fMRI literature is that healthy old adults present higher activity levels in some brain regions during the performance of cognitive tasks. This finding is usually interpreted as a compensatory mechanism. More recent approaches have focused on the study of functional connectivity, mainly derived from resting state fMRI, and have concluded that the higher levels of activity coexist with disrupted connectivity. In this review, we aim to provide a state-of-the-art description of the usefulness and the interpretations of functional brain connectivity in the context of HA. We first give a background that includes some basic aspects and methodological issues regarding functional connectivity. We summarize the main findings and the cognitive models that have been derived from task-activity studies, and we then review the findings provided by resting state functional connectivity in HA. Finally, we suggest some future directions in this field of research. A common finding of the studies included is that older subjects present reduced functional connectivity compared to young adults. This reduced connectivity affects the main brain networks and explains age-related cognitive alterations. Remarkably, the default mode network appears as a highly compromised system in HA. Overall, the scenario given by both activity and connectivity studies also suggests that the trajectory of changes during task may differ from those observed during resting-state. We propose that the use of complex modeling approaches studying effective connectivity may help to understand context-dependent functional reorganizations in the aging process.