This study aimed to investigate the effects of different levels of soil- and plant-associated bacterial diversity on the rates of litter decomposition, and bacterial community dynamics during its early phases. We performed an incubation experiment where soil bacterial diversity (but not abundance) was manipulated by autoclaving and reinoculation. Natural or autoclaved maize leaves were applied to the soils and incubated for 6 weeks. Bacterial diversity was assessed before and during litter decomposition using 16S rRNA gene metabarcoding. We found a positive correlation between litter decomposition rates and soil bacterial diversity. The soil with the highest bacterial diversity was dominated by oligotrophic bacteria including Acidobacteria, Nitrospiraceae, and Gaiellaceae, and its community composition did not change during the incubation. In the less diverse soils, those taxa were absent but were replaced by copiotrophic bacteria, such as Caulobacteraceae and Beijerinckiaceae, until the end of the incubation period. SourceTracker analysis revealed that litter-associated bacteria, such as Beijerinckiaceae, only became part of the bacterial communities in the less diverse soils. This suggests a pivotal role of oligotrophic bacteria during the early phases of litter decomposition and the predominance of copiotrophic bacteria at low diversity.