Inclusion body myositis (IBM) is a slowly progressive muscle weakness of distal and proximal muscles, which is diagnosed by clinical and histopathological criteria. Imaging biomarkers are inconsistently used and do not follow international standardized criteria. We conducted a systematic review and meta-analysis to investigate the diagnostic value of muscle ultrasound (US) in IBM compared to healthy controls. A systematic search of PubMed/MEDLINE, Scopus and Web of Science was performed. Articles reporting the use of muscle ultrasound in IBM, and published in peer-reviewed journals until 11 September 2021, were included in our study. Seven studies were included, with a total of 108 IBM and 171 healthy controls. Echogenicity between IBM and healthy controls, which was assessed by three studies, demonstrated a significant mean difference in the flexor digitorum profundus (FDP) muscle, which had a grey scale value (GSV) of 36.55 (95% CI, 28.65–44.45, p < 0.001), and in the gastrocnemius (GC), which had a GSV of 27.90 (95% CI 16.32–39.48, p < 0.001). Muscle thickness in the FDP showed no significant difference between the groups. The pooled sensitivity and specificity of US in the differentiation between IBM and the controls were 82% and 98%, respectively, and the area under the curve was 0.612. IBM is a rare disease, which is reflected in the low numbers of patients included in each of the studies and thus there was high heterogeneity in the results. Nevertheless, the selected studies conclusively demonstrated significant differences in echogenicity of the FDP and GC in IBM, compared to controls. Further high-quality studies, using standardized operating procedures, are needed to implement muscle ultrasound in the diagnostic criteria.