Postdoctoral researcher, University of California, San Diego
Practice effects from repeat testing can result in underdiagnosis of mild cognitive impairment
True measures of cognitive decline across the lifespan must be obtained through longitudinal testing, but age-based norms do not account for practice effects (PEs) of repeat testing. We hypothesized that not accounting for PEs could result in under-diagnosis of mild cognitive impairment (MCI) at follow-up assessments.
Methods: During wave 1 of the Vietnam Era Twin Study of Aging, 1220 adults (mean age=55.88) completed a neuropsychological testing. Approximately 5.6 years later, 995 individuals returned for wave 2, and 170 age-matched attrition replacement (AR) subjects were tested for the first time. Groups were matched for early adult general cognitive ability by regressing out scores on a cognitive ability test each individual completed around age 20. PEs were calculated according to Rönnlund et al. (2005). The diﬀerence (D) between wave 2 scores of returnees and AR subjects is the sum of attrition eﬀects (A) and PEs. The attrition eﬀect is calculated as the diﬀerence in wave 1 scores of returnees compared to the entire group. The PE is therefore the diﬀerence (D) minus the attrition eﬀect (A). PEs were then subtracted from returnees’ wave 2 test scores. Significance of each PE was determined through permutation testing. MCI was defined according to modified Jak/Bondi criteria. Individuals scoring >1.5 SDs below age- and education-adjusted normative means on ≥2 tests within a cognitive domain were considered impaired. We have validated these diagnoses against Alzheimer’s disease polygenic risk scores. Results: Twenty-three of 65 cognitive scores demonstrated significant PEs. Among returnees, adjusting for PEs resulted in an increase in the conversion rate of MCI diagnoses from 4.6% to 9% and prevalence from 9% to 15%. Not adjusting for PEs also overestimated the number of individuals with MCI reverting to cognitively normal. Conclusions: Cognitive tests can be affected by PEs over intervals of more than 5 years. Repeated testing may mask actual declines in performance and result in under-diagnosis of MCI when cognitive impairment is based on normative means. These findings suggest the importance of accounting for PEs through inclusion of AR subjects in longitudinal studies of aging and dementia, particularly with respect to early identification.
Abstract: Longitudinal comparisons of neurocognitive functioning often reveal stability or age-related increases in performance among adults under about 60 years of age. Because nearly monotonic declines with increasing age are typically evident in cross-sectional comparisons, there is a discrepancy in the inferred age trends based on the two types of comparisons. The current research investigated the role of practice effects in longitudinal comparisons on the discrepancy.Longitudinal data over an average interval of 2.5 years were available on five abilities (i.e., reasoning, spatial visualization, episodic memory, perceptual speed, vocabulary) in a sample of 1,616 adults ranging from 18 to over 80 years of age. Practice effects were estimated from comparisons of the performance of people of the same age tested for either the first or second time, after adjusting for the possibility of selective attrition.Increased age was associated with significantly more negative longitudinal changes with each ability. All of the estimated practice effects were positive, but they varied in magnitude across neurocognitive abilities and as a function of age. After adjusting for practice effects the longitudinal changes were less positive at younger ages and slightly less negative at older ages.It was concluded that some, but not all, of the discrepancy between cross-sectional and longitudinal age trends in neurocognitive functioning is attributable to practice effects positively biasing the longitudinal trends. These results suggest that the neurobiological substrates of neurocognitive functioning may change across different periods in adulthood.
Pub.: 02 Sep '10, Pinned: 19 Jun '17
Abstract: In neuropsychological assessment, and many areas of research, it is common for the same test to be administered on more than one occasion to measure change. Measured changes are presumed to reflect true changes in the construct being measured by the test; for example, cognitive changes due to processes such as aging, advancing neurological disease, or treatment interventions. However, practice effects, defined as score increases due to factors such as memory for specific test items, learned strategies, or test sophistication, complicate the interpretation of change. This review presents meta-analyses of nearly 1600 individual effect sizes representing changes in mean-level performance on tests commonly used to assess core domains of neuropsychological function, with the goal of quantitatively summarizing the magnitude of practice effects on such tests. The use of alternate forms, the ages of participants, clinical diagnoses of study participants, and length of the test-retest interval were associated with the magnitude of change in many cases. These findings have important implications for the practice of clinical neuropsychology, as well as for research applications, and highlight the need for practice effects to be taken into account in interpreting change across time with multiple measurements.
Pub.: 01 May '12, Pinned: 19 Jun '17
Join Sparrho today to stay on top of science
Discover, organise and share research that matters to you