Background: Tuberculosis (TB) and sarcoidosis are two common granulomatous diseases involving lymph nodes. Differential diagnosis is not always easy because pathogen demonstration in tuberculosis is not always possible and both diseases share clinical, radiological and histological patterns. The aim of our study was to identify factors associated with each diagnosis and set up a predictive score for TB. Methods: All cases of lymph node tuberculosis and sarcoidosis were retrospectively reviewed. Demographics, clinical characteristics, laboratory and imaging data, and microbiological and histological results were collected and compared. Results: Among 441 patients screened, 192 patients were included in the final analysis. The multivariate analysis showed that weight loss, necrotic granuloma, normal serum lysozyme level and hypergammaglobulinemia were significantly associated with TB. A risk score of TB was built based on these variables and was able to discriminate TB versus sarcoidosis with an AUC of 0.85 (95% CI: 0.79–0.91). Using the Youden’s J statistic, its most discriminant value (−0.36) was associated with a sensitivity of 80% and a specificity of 75%. Conclusions: We developed a score based on weight loss, necrotic granuloma, normal serum lysozyme level and hypergammaglobulinemia with an excellent capacity to discriminate TB versus sarcoidosis. This score needs still to be validated in a multicentric prospective study.