This paper carries out a numerical simulation investigating three different modulation formats: carrier suppressed return to zero (CSRZ), modified duobinary return to zero (MDRZ) and return to zero (RZ). The purpose of this investigation is to find the optimum modulation format for the hybrid optical code division multiple accesses–dense wavelength division multiplexing (optical CDMA/DWDM) system with the implementation of an electro-optic phase modulator (EOPM) at a data rate of 40 Gbps per channel, with a transmitted power of 22 dBm and transmission distance of 105.075 km. The results revealed that CSRZ was superior to MDRZ and RZ and was more tolerant to optical fiber nonlinearity. Furthermore, unlike the DWDM systems, the performance of the proposed hybrid system based RZ format was better than the performance of MDRZ. Hence, the CSRZ modulation format is the best candidate for the optical CDMA/DWDM with EOPM module due to its high performance.