Luminescence quenching via non-radiative recombination channels limits the efficiency of optical materials such as phosphors and scintillators and therefore has implications for conversion efficiency and device lifetimes. In materials such as Ce-doped yttrium aluminum garnet (YAG:Ce), quenching shows a strong dependence on both temperature and activator concentration, limiting the ability to fabricate high-intensity white-light emitting diodes with high operating temperatures. Here, we reveal by means of first-principles calculations an efficient recombination mechanism in YAG:Ce that involves oxygen vacancies and gives rise to thermally activated concentration quenching. We demonstrate that the key requirements for this mechanism to be active are localized states with strong electron-phonon coupling. These conditions are commonly found for intrinsic defects such as anion vacancies in wide band-gap materials. The present findings are therefore relevant to a broad class of optical materials and shine light on thermal quenching mechanisms in general.