The aim of this study was to compare the efficacy and safety of focused low-intensity pulsed ultrasound (FLIPUS) with pulsed shortwave diathermy (PSWD) in subjects with painful knee osteoarthritis (OA). In a prospective randomized trial, 114 knee OA patients were randomly allocated to receive FLIPUS or PSWD therapy. The primary outcome was the change from baseline in the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) total scores. Secondary outcomes included the numerical rating scale (NRS) for pain assessment, time up and go (TUG) test, active joint range of motion (ROM) test, and Global Rating of Change (GRC) scale. Data were collected at baseline, 12 days, 12 weeks and 24 weeks. Patients receiving FLIPUS therapy experienced significantly greater improvements in the WOMAC total scores than patients receiving PSWD therapy at 12 days (mean difference, − 10.50; 95% CI − 13.54 to − 7.45; P = 0.000). The results of the NRS, TUG test, ROM test and GRC scale showed that participants treated with FLIPUS reported less pain and better physical function and health status than those treated with PSWD at 12 days (P = 0.011, P = 0.005, P = 0.025, P = 0.011, respectively). Furthermore, patients in the FLIPUS group showed significant improvements in the WOMAC total scores and NRS scores at 12 weeks (mean difference, − 7.57; 95% CI − 10.87 to − 4.26; P = 0.000 and − 1.79; 95% CI − 2.11 to − 1.47, respectively) and 24 weeks (mean difference, − 6.96; 95% CI − 10.22 to − 3.71; P = 0.000 and − 1.37; 95% CI − 1.64 to − 0.96; P = 0.000, respectively) of follow-up. There were no adverse events during or after the interventions in either group. This study concluded that both FLIPUS and pulsed SWD are safe modalities, and FLIPUS was more effective than PSWD in alleviating pain and in improving dysfunction and health status among subjects with knee OA in the short term.Trial registration: Chinese Clinical Trial Registry, ChiCTR2000032735. Registered 08/05/2020, http://www.chictr.org.cn/showproj.aspx?proj=53413.