Stems are the important residues of Trapa quadrispinosa Roxb., which are abundant in phenolic compounds. Ultrasonic-assisted enzymatic extraction (UAEE) is confirmed as a novel extraction technology with main advantages of enhancing extraction yield and physiological activities of the extracts from various plants. In this study, UAEE was applied to obtain the highest yield of phenolic content, strongest antioxidant, and antitumor activities and to optimize the extraction conditions using response surface methodology (RSM). The extracts from the stems of T. quadrispinosa were characterized by determination of their antioxidant activities through 2,2-azinobis(3-ethylbenzthiazoline)-6-sulfonic acid (ABTS), 1,1-Diphenyl-2-picrylhydrazxyl (DPPH) radical scavenging, total antioxidant capacity (TAC), ferric reducing antioxidant capacity (FRAC) methods and of their antitumor activity by MTT method. The selected key independent variables were cellulase concentration (X₁: 1.5%–2.5%), extraction time (X₂: 20–30 min) and extraction temperature (X₃: 40–60 °C). The optimal extraction conditions for total phenolic content (TPC) value of the extracts were determined as 1.74% cellulase concentration, 25.5 min ultrasonic extraction time and 49.0 °C ultrasonic temperature. Under these conditions, the highest TPC value of 53.6 ± 2.2 mg Gallic acid equivalent (GAE)/g dry weight (DW) was obtained, which agreed well with the predicted value (52.596 mg GAE/g·DW. Furthermore, the extracts obtained from UAEE presented highest antioxidant activities through ABTS, DPPH, TAC and FRAC methods were of 1.54 ± 0.09 mmol Trolox equivalent (TE)/g·DW; 1.45 ± 0.07 mmol·TE/g·DW; 45.2 ± 2.2 mg·GAE/g·DW; 50.4 ± 2.6 μmol FeSO₄ equivalent/g·DW and lowest IC₅₀ values of 160.4 ± 11.6 μg/mL, 126.1 ± 10.8 μg/mL, and 178.3 ± 13.1 μg/mL against Hela, HepG-2 and U251 tumor cells, respectively. The results indicated that the UAEE was an efficient alternative to improve extraction yield and enhance the antioxidant and antitumor activities of the extracts. The phenolic extracts from the stems of T. quadrispinosa had significant antioxidant and antitumor activities, which could be used as a source of potential antioxidant and antitumor agents.