Purpose The objective of this study was to evaluate the psychometric properties of the Constant–Murley Score (CMS) in various shoulder pathologies, based on a systematic review and expert standardized evaluations. Methods A systematic review was performed in MEDLINE and EMBASE databases. Titles and abstracts were reviewed and finally the included articles were grouped according to patients' pathologies. Two expert evaluators independently assessed the CMS properties of reliability, validity, responsiveness to change, interpretability and burden score in each group, using the EMPRO (Evaluating Measures of Patient Reported Outcomes) tool. The CMS properties were assessed per attribute and overall for each considered group. Only the concept and measurement model was assessed globally. Results Five individual pathologies (i.e. subacromial, fractures, arthritis, instability and frozen shoulder) and two additional groups (i.e. various pathologies and healthy subjects) were considered. Overall EMPRO scores ranged from 58.6 for subacromial to 30.6 points for instability. Responsiveness to change was the only quality to obtain at least 50 points across all groups, but for frozen shoulder. Insufficient information was obtained in relation to the concept and measurement model and great variability was seen in the other evaluated attributes. Conclusions The current evidence does not support the CMS as a gold standard in shoulder evaluation. Its use is advisable for subacromial pathology; but data are inconclusive for other shoulder conditions. Prospective studies exploring the psychometric properties of the scale, particularly for fractures, arthritis, instability and frozen shoulder are needed. Level of evidence Systematic review.