The biodiversity–productivity relationship (BPR) is crucial for understanding the consequences of biodiversity loss; yet, how this relationship evolves over time in natural ecosystems remains unclear. Several long-term biodiversity experiments with artificially assembled communities suggest that the BPR escalates over time, however, whether this pattern applies to natural ecosystems has not been explored. Here, we present the results of a 15-y removal experiment in the Inner Mongolia grassland, where gradients of plant functional group (PFG) richness and species richness were created through targeted removals from 2006–2009, followed by natural recolonization from 2010 onward. We found that both the PFG richness-based and species richness-based BPRs diminished gradually during the removal period (years 2–4) and decoupled in 9 out of 12 y after the cessation of removal (years 5–16), suggesting that the negative impacts of biodiversity loss do not necessarily escalate over time; rather, they may decrease. We identified that the temporal shift in BPRs resulted first from compensatory growth of remaining species and subsequently from new colonization by external populations. These results highlight that the impacts of biodiversity loss in natural ecosystems are strongly modified by assembly processes such as compensation and colonization. Therefore, models predicting the ecosystem consequences of biodiversity loss should consider these mechanisms. Ignoring them may lead to an overestimation of the long-term impacts of biodiversity loss in natural communities, particularly where surrounding landscapes harbor species pools capable of recolonization.