Coffee consumption is increasing yearly, and so is the quantity of spent coffee grounds, a by-product that still contains bioactive compounds. The goal of this work is to evaluate the lipid fractions (LFs) and polyphenolic fractions (PFs) from roasted coffee grounds (RCG; two varieties of Ugandan origin: Arabica and Robusta) and from the corresponding spent coffee grounds (SCG) extracted with solvents of opposite polarities. A qualitative and quantitative analysis of the extracts was performed using GC-FID, ¹H-NMR, Folin–Ciocalteu, HPLC-PDA, DPPH and antibacterial activity methods. A high-quality LF, coffee oil (91.82 g triacylglycerides/100 g LF, 49.85 mol% saturated fatty acids and 50.15 mol% unsaturated fatty acid chains, 8.98 mg GAE/g LF, 20.91% DPPH inhibition), with high extraction yield (10.34 g/100 g SCG), can be recovered from SCG of the Ugandan Arabica variety. In addition, SCG of both Ugandan Arabica and Robusta varieties are promising resources for the recovery of high-quality PF (12.30 and 7.85 mg GAE/g PF, 0.684 and 0.837 mg/g PF chlorogenic acids, 85.17% and 73.95% DPPH inhibition) with good extraction yields (9.35 and 5.63 g/100 g SCG) and with antimicrobial activities that remain after the coffee brewing step. These results highlight SCG of Ugandan origin as a promising resource of bioactive compounds for further valorization.