Almost two years after remdesivir was approved and extensively used in numerous clinical studies for the treatment of COVID-19 patients, there is still no clear recommendation for the time and phase of the disease of remdesivir administration. This retrospective observational study included adults (≥18 years) with severe COVID-19, radiologically confirmed pneumonia, a need for supplemental oxygen and an interval from symptom onset to enrolment of 10 days or less. All patients were treated with remdesivir for 5 to 10 days, or with clinical improvement within that period. The primary goal was the outcome in patients treated with remdesivir during the early stage of the disease considering the different disease severity. The median time from symptom onset to treatment was 8.4 days (3–10). Clinical improvements and good outcomes were observed in 104 of 137 patients (75.9%); 33 (24.1%) of 137 patients died. Subgroup analyses showed that the mortality rate was significantly lower in moderately ill patients (3 out of 51 patients; 5.9%) than in the group of severely/critically ill patients (30 out of 86 patients; 34.8%; p < 0.005). Older age, rise of CRP and CT score were shown to be significant predictors of disease outcome. Overall, remdesivir was well tolerated, and the treatment was discontinued in only four patients. The results of this observational study in 137 patients with different disease severity contribute to the attitude concerning remdesivir administration in the early stage of COVID-19, at least in moderately ill patients with a high risk of progression, before the transition to a more severe stage.