In this paper, fatigue analysis of oil offloading lines (OOLs) in the floating production storage and offloading (FPSO) catenary anchor leg mooring (CALM) buoy offloading system under wave and current loads in the West Africa Sea area is carried out by the numerical simulation method. The hydrodynamic coupling response is calculated, and fatigue damage is analyzed. Firstly, the numerical model is verified by comparison with the experimental results. Then, according to the environmental statistics in West Africa, the influence of various parameters on the fatigue damage of OOLs is analyzed, including tension characteristics, wave parameters, and structural parameters. Additionally, the effect of current load is studied. Results show that accumulated fatigue damage mainly occurs near the CALM buoy and is mainly caused by the 0° wind wave. Appropriately reducing the cover length of buoyancy material and increasing the wall thickness can reduce fatigue damage. Moreover, the effect of the shuttle tanker can increase the fatigue damage of the OOL near the CALM buoy by about 1.5 times, and the effect of vortex-induced vibration can increase the fatigue damage of the OOL in the middle part by up to 5–10 times.