Myeloid derived suppressor cells (MDSCs), which suppress anti-tumor or anti-viral immune responses, are expanded in the peripheral blood and tissues of patients/animals with cancer or viral infectious diseases. We here show that in chronic SIV infection of Indian rhesus macaques, the frequency of MDSCs in the bone marrow (BM) was paradoxically and unexpectedly decreased, but increased in peripheral blood. Reduction of BM MDSCs was found in both CD14+MDSC and Lin-CD15+MDSC subsets. The reduction of MDSCs correlated with high plasma viral loads and low CD4+ T cell counts, suggesting that depletion of BM MDSCs was associated with SIV/AIDS disease progression. Of note, in SHIVSF162P4-infected macaques, which naturally control viral replication within a few months of infection, the frequency of MDSCs in the bone marrow was unchanged. To investigate the mechanisms by which BM MDSCs were reduced during chronic SIV infection, we tested several hypotheses: depletion due to viral infection, alterations in MDSC trafficking, and/or poor MDSC replenishment. We found that the possible mobilization of MDSCs from BM to peripheral tissues and the slow self-replenishment of MDSCs in the BM, along with the viral infection-induced depletion, all contribute to the observed BM MDSC reduction. We first demonstrate MDSC SIV infection in vivo. Correlation between BM CD14+MDSC reduction and CD8+ T cell activation in tissues is consistent with decreased immune suppression by MDSCs. Thus, depletion of BM MDSCs may contribute to the pathologic immune activation during chronic SIV infection and by extension HIV infection. Both cancer and infectious diseases including HIV/AIDS lead to the accumulation of myeloid-derived suppressor cells (MDSCs), which can effectively suppress anti-tumor and anti-viral T cell responses to dampen protective immunity. Using a macaque model, we found unexpectedly that the MDSCs in bone marrow (BM) decreased after chronic simian immunodeficiency virus (SIV) infection compared with healthy controls. This was in sharp contrast to the general increase of MDSCs observed in BM during cancer and other infectious/inflammatory diseases, and also contrary to the MDSC expansion in HIV/SIV-infected PBMCs. We further demonstrated that the loss of MDSCs in the bone marrow was associated with the progression to AIDS disease. Investigating the mechanisms by which the MDSCs were decreased in the SIV-infected bone marrow, we found that the possible mobilization of MDSCs from bone marrow to peripheral tissues and the slow self-replenishment of MDSCs in the bone marrow, along with the viral infection-induced depletion, all contribute to the observed bone marrow MDSC reduction. Indeed, this is the first demonstration to our knowledge of SIV infection of MDSCs in vivo. Because of the suppressive nature of the MDSCs, the CD8+ T cells might not be effective in killing the virally infected MDSCs. It is tempting to speculate that MDSCs may constitute latent reservoirs. Overall, our data showed that MDSCs act as a double-edged sword in HIV/SIV-infection, and the decrease of MDSCs in bone marrow after SIV infection could serve as an indicator of immune regulatory exhaustion and also contribute to the observed immune hyperactivation seen in HIV/AIDS.