Automatic modulation classification (AMC) is an essential technique in intelligent receivers of non-cooperative communication systems such as cognitive radio networks and military applications. This article proposes a robust automatic modulation classification model based on a new architecture of a convolutional neural network (CNN). The basic building convolutional blocks of the proposed model include asymmetric kernels organized in parallel combinations to extract more meaningful and powerful features from the raw I/Q sequences of the received signals. These blocks are connected via skip connection to avoid vanishing gradient problems. The experimental results reveal that the proposed model performs well in classifying nine different modulation schemes simulated with different real wireless channel impairments, including AWGN, Rician multipath fading, and clock offset. The performance of the proposed system systems shows that it outperforms its best rivals from the literature in recognizing the modulation type. The proposed CNN architecture remarkably improves classification accuracy at low SNRs, which is appropriate in realistic scenarios. It achieves 86.1% accuracy at −2 dB SNR. Furthermore, it reaches an accuracy of 96.5% at 0 dB SNR and 99.8% at 10 dB SNR. The proposed architecture has strong feature extraction abilities that can effectively recognize 16QAM and 64QAM signals, the challenging modulation schemes of the same modulation family, with an overall average accuracy of 81.02%.