Recent studies suggest that the dietary intake of human milk oligosaccharides (HMOs) provides health benefits from infancy up to adulthood. Thus far, beneficial changes in the adult gut microbiome have been observed at oral doses of 5–20 g/day of HMOs. Efficacy of lower doses has rarely been tested. We assessed four HMO molecular species—2′Fucosyllactose (2′FL), Lacto-N-neotetraose (LNnT), 3′Sialyllactose (3′SL), and 6′Sialyllactose (6′SL)—at predicted doses from 0.3 to 5 g/day for 6-year-old children and adults (n = 6 each), using ex vivo SIFR^® technology (Cryptobiotix, Ghent, Belgium). This technology employing bioreactor fermentation on fecal samples enables us to investigate microbial fermentation products that are intractable in vivo given their rapid absorption/consumption in the human gut. We found that HMOs significantly increased short-chain fatty acids (SCFAs), acetate, propionate (in children/adults), and butyrate (in adults) from predicted doses of 0.3–0.5 g/day onwards, with stronger effects as dosing increased. The fermentation of 6′SL had the greatest effect on propionate, LNnT most strongly increased butyrate, and 2′FL and 3′SL most strongly increased acetate. An untargeted metabolomic analysis revealed that HMOs enhanced immune-related metabolites beyond SCFAs, such as aromatic lactic acids (indole-3-lactic acid/3-phenyllactic acid) and 2-hydroxyisocaproic acid, as well as gut–brain-axis-related metabolites (γ-aminobutyric acid/3-hydroxybutyric acid/acetylcholine) and vitamins. The effects of low doses of HMOs potentially originate from the highly specific stimulation of keystone species belonging to, for example, the Bifidobacteriaceae family, which had already significantly increased at doses of only 0.5 g/day LNnT (adults) and 1 g/day 2′FL (children/adults).