Rising health concerns regarding chronic diseases call for exploring natural sources of antioxidants and factors that influence their activity. This study evaluated the diversity of 112 sesame germplasms from Africa and Asia based on ten agronomic traits (seven quantitative and three qualitative), two antioxidant activities (ABTS and DPPH radical scavenging activities), and the content of one metabolite (TPC). TPC, DPPH, and ABTS were in the ranges of 4.98–87.88 µg GAE/mg DE, 3.97–46.23 µg AAE/mg DE, and 3.42–176.01 µg TE/mg DE, respectively. Statistical analyses revealed significant variations in agronomic traits, TPC, and antioxidant activities among the sesame germplasms (p < 0.05). Furthermore, the individual and interaction effects of seed color and the continent of origin on the levels of the quantitative traits, TPC, ABTS, and DPPH were analyzed, and the correlation among the traits was further evaluated. Diversity in TPC, ABTS, and DPPH was significantly associated with seed color and most of the quantitative agronomic traits (p < 0.05) but not with continent of origin. Principal component analysis revealed TPC, ABTS, DPPH, and five quantitative traits as the most discriminant traits. In general, six sesame accessions with high TPC and antioxidant activities (IT194356, IT170094, IT29971, IT185998, IT104246, and IT169623) as well as important agronomic traits were identified and, hence, could be used for developing improved sesame varieties.