The genus Vitex is also known as a chaste tree, in which it is a large shrub native to the tropical and subtropical regions of the world. A diverse range of species is distributed throughout Southern Europe, the Mediterranean, and Central Asia. The Vitex tree, including its leaves and fruits, has been used for herbal remedies in the form of pastes, decoctions, and dried fruits since ancient times. This article aimed to prepare a comprehensive review of traditional uses and secondary metabolites derived from Vitex sp., including the chemical compounds, biological activities, application of Vitex in human clinical trials, toxicology and safety, marketed products, and patents. The scientific findings were obtained using a number of search engines and databases, including Google Scholar, PMC, and ScienceDirect. Vitex species are well known in pharmacology to have medicinal values, such as anti-inflammatory, antibacterial, antifungal, antimicrobial, antioxidant, and anticancer properties. Previous studies reported that some species are proven to be effective in treating diseases, such as diabetes, and improving female health. A total of 161 compounds from different Vitex species are reported, covering the literature from 1982 to 2022. A chemical analysis report of various studies identified that Vitex exhibited a wide range of phytoconstituents, such as iridoid, diterpenoid, ecdysteroid, and flavonoid and phenolic compounds. Apart from that, the review will also discuss the application of Vitex in human clinical trials, toxicology and safety, marketed products, and patents of the genus. While the extracts of the genus have been made into many commercial products, including supplements and essential oils, most of them are made to be used by women to improve menstrual conditions and relieve premenstrual syndrome. Among the species, Vitex agnus-castus L. is the only one that has been reported to undergo clinical trials, mainly related to the use of the genus for the treatment of mastalgia, menstrual bleeding problems, amenorrhea, menorrhagia, luteal insufficiency, and premenstrual syndrome. Overall, the review addresses recent therapeutic breakthroughs and identifies research gaps that should be explored for prospective research work.