Although gut dysbiosis is associated with cow’s milk allergy (CMA), causality remains uncertain. This study aimed to identify specific bacterial signatures that influence the development and outcome of the disease. We also investigated the effect of hypoallergenic formula (HF) consumption on the gut microbiome of milk-allergic children. 16S rRNA amplicon sequencing was applied to characterize the gut microbiome of 32 milk-allergic children aged 5–12 years and 36 age-matched healthy controls. We showed that the gut microbiome of children with CMA differed significantly from that of healthy children, regardless of whether they consumed cow’s milk. Compared to that of healthy cow’s milk consumers, it was depleted in Bifidobacterium, Coprococcus catus, Monoglobus, and Lachnospiraceae GCA-900066575, while being enriched in Oscillibacter valericigenes, Negativibacillus massiliensis, and three genera of the Ruminococcaceae family. Of these, only the Ruminococcaceae taxa were also enriched in healthy children not consuming cow’s milk. Furthermore, the gut microbiome of children who developed tolerance and had received an HF was similar to that of healthy children, whereas that of children who had not received an HF was significantly different. Our results demonstrate that specific gut microbiome signatures are associated with CMA, which differ from those of dietary milk elimination. Moreover, HF consumption affects the gut microbiome of children who develop tolerance.