Early-life gut microbiota plays a role in determining the health and risk of developing diseases in later life. Various perinatal factors have been shown to contribute to the development and establishment of infant gut microbiota. One of the important factors influencing the infant gut microbial colonization and composition is the mode of infant feeding. While infant formula milk has been designed to resemble human milk as much as possible, the gut microbiome of infants who receive formula milk differs from that of infants who are fed human milk. A diverse microbial population in human milk and the microbes seed the infant gut microbiome. Human milk contains nutritional components that promote infant growth and bioactive components, such as human milk oligosaccharides, lactoferrin, and immunoglobulins, which contribute to immunological development. In an attempt to encourage the formation of a healthy gut microbiome comparable to that of a breastfed infant, manufacturers often supplement infant formula with prebiotics or probiotics, which are known to have a bifidogenic effect and can modulate the immune system. This review aims to elucidate the roles of human milk and formula milk on infants’ gut and health.