Background/Objectives: Return to work (RTW) after an industrial accident is crucial for an individual’s well-being and socioeconomic recovery. This study investigated factors influencing RTW among workers who sustained physical or brain lesion-related disabilities following industrial accidents. Methods: Using five-year panel data (2018–2022) from the Panel Study of Workers’ Compensation Insurance of South Korea, we analyzed 340 individuals with physical or brain lesion-related disabilities sustained from industrial accidents. We used logistic regression models to identify factors associated with RTW and return to employed (RTE) status. Results: The RTW and non-RTW groups comprised 160 and 180 participants, respectively. Factors associated with non-RTW included female sex (adjusted odds ratio [aOR], 0.33; 95% confidence interval [CI], 0.13–0.86; p = 0.023), injury caused by disease (aOR, 0.18; 95% CI, 0.05–0.66; p = 0.010), long recovery periods (aOR, 0.27; 95% CI, 0.10–0.72; p = 0.009), low self-confidence (aOR, 0.16; 95% CI, 0.07–0.35; p < 0.001), and older age (aOR, 0.08; 95% CI, 0.02–0.34; p = 0.001). Workers with no blood pressure problems (aOR, 2.21; 95% CI, 1.11–4.38; p = 0.024) and longer employment durations (aOR, 3.84; 95% CI, 1.15–12.81; p = 0.029) had a higher chance of RTW. Similar factors were associated with RTE, with more emphasis on older age, long recovery periods, low self-confidence, and injury caused by disease. Conclusions: Our findings contribute to developing targeted support services and informing policy decisions to improve RTW for workers with physical or brain lesion-related disabilities caused by industrial accidents.