Natural compounds have been recognized as valuable sources for anticancer drug development. In this work, different parts from Momordica cochinchinensis Spreng were selected to perform cytotoxic screening against human prostate cancer (PC-3) cells. Chromatographic separation and purification were performed for the main constituents of the most effective extract. The content of the fatty acids was determined by Gas Chromatography-Flame Ionization Detector (GC–FID). Chemical structural elucidation was performed by spectroscopic means. For the mechanism of the apoptotic induction of the most effective extract, the characteristics were evaluated by Hoechst 33342 staining, sub-G1 peak analysis, JC-1 staining, and Western blotting. As a result, extracts from different parts of M. cochinchinensis significantly inhibited cancer cell viability. The most effective stem extract induced apoptosis in PC-3 cells by causing nuclear fragmentation, increasing the sub-G1 peak, and changing the mitochondrial membrane potential. Additionally, the stem extract increased the pro-apoptotic (caspase-3 and Noxa) mediators while decreasing the anti-apoptotic (Bcl-xL and Mcl-1) mediators. The main constituents of the stem extract are α-spinasterol and ligballinol, as well as some fatty acids. Our results demonstrated that the stem extract of M. cochinchinensis has cytotoxic and apoptotic effects in PC-3 cells. These results provide basic knowledge for developing antiproliferative agents for prostate cancer in the future.