Anthropogenic drivers of environmental change often have multiple effects, including changes in biodiversity, species composition, and ecosystem functioning. It remains unknown whether such shifts in biodiversity and species composition may, themselves, be major contributors to the total, long-term impacts of anthropogenic drivers on ecosystem functioning. Moreover, although numerous experiments have shown that random losses of species impact the functioning of ecosystems, human-caused losses of biodiversity are rarely random. Here we use results from long-term grassland field experiments to test for direct effects of chronic nutrient enrichment on ecosystem productivity, and for indirect effects of enrichment on productivity mediated by resultant species losses. We found that ecosystem productivity decreased through time most in plots that lost the most species. Chronic nitrogen addition also led to the nonrandom loss of initially dominant native perennial C4 grasses. This loss of dominant plant species was associated with twice as great a loss of productivity per lost species than occurred with random species loss in a nearby biodiversity experiment. Thus, although chronic nitrogen enrichment initially increased productivity, it also led to loss of plant species, including initially dominant species, which then caused substantial diminishing returns from nitrogen fertilization. In contrast, elevated CO2 did not decrease grassland plant diversity, and it consistently promoted productivity over time. Our results support the hypothesis that the long-term impacts of anthropogenic drivers of environmental change on ecosystem functioning can strongly depend on how such drivers gradually decrease biodiversity and restructure communities.