Adherence to a Mediterranean diet is associated with higher cognitive function and reduced risk of dementia in Mediterranean populations. However, few studies have investigated the association between Mediterranean diet adherence and cognition in populations outside of the Mediterranean basin. Furthermore, it is currently unknown whether the association between Mediterranean diet adherence and cognitive function differs between middle-aged and older individuals. Cross-sectional ( = 894) and longitudinal ( = 530) multivariable analyses were undertaken using data from community-dwelling adults from the Maine-Syracuse Longitudinal Study (MSLS). Mediterranean diet adherence was measured by applying a literature-based Mediterranean diet score to food frequency questionnaire data. Cognitive function was assessed with a battery of tests and composites scores were computed for global cognitive function, Visual-Spatial Organisation and Memory, verbal memory, working memory, scanning and tracking and abstract reasoning. No cross-sectional associations between Mediterranean diet adherence and cognitive function were detected. Over a period of five years, higher adherence to a Mediterranean diet was associated with improvements in Global Cognitive Function, Visual-Spatial Organisation and Memory and scanning and tracking in participants ≥70 years. No significant longitudinal associations were observed for participants <70 years. Our findings suggest that higher adherence to a Mediterranean diet is associated with better cognitive performance, and therefore less cognitive decline, in older but not middle-aged individuals.