The association between nuts intake and cognitive function is inconclusive. We aimed to investigate the association between habitual nuts consumption and cognition among Qatari adults. Data from 1000 participants aged >20 years who attended Qatar Biobank (QBB) were used. Nuts consumption was assessed by a food frequency questionnaire (FFQ). Blood samples were measured for magnesium, lipids and glucose. Mean reaction time (MRT) was used as an indicator of cognitive function. Linear regression was used to assess the association. A total of 21.1% of the participants reported consuming nuts ≥4–6 times/week (high consumption) while 40.2% reported consuming ≤1 time/month (low consumption). The mean MRT was 715.6 milliseconds (SD 204.1). An inverse association was found between nuts consumption and MRT. Compared to those with a low consumption, high consumption of nuts had a regression coefficient of −36.9 (95% CI −68.1 to −5.8) after adjusting for sociodemographic and lifestyle factors. The inverse association between nuts and MRT was mainly seen among those >50 years. There was an interaction between nuts consumption and hypertension. The association between nuts consumption and MRT was not mediated by hypertension, diabetes, or serum magnesium. Habitual higher consumption of nuts is positively associated with cognitive function, especially among old adults.