Deep learning (DL) has recently aroused substantial concern due to its successful implementations in many fields. Currently, there are few studies on the applications of DL in the automatic modulation classification (AMC), which plays a critical role in non-cooperation communications. Besides, most previous work ignores the feature interaction and only considers spatial or temporal attributes of signals. Combining the advantages of the convolutional neural network (CNN) and the long short-term memory (LSTM), this paper addresses the AMC using CNN-LSTM based dual-stream structure, which efficiently explores the feature interaction and the spatial-temporal properties of raw complex temporal signals. Specifically, a preprocessing step is first implemented to convert signals into the temporal in-phase/ quadrature (I/Q) format and the amplitude/phase (A/P) representation, which facilitates the acquirement of more effective features for classification. To extract features from each signal pattern, each stream is composed of CNN and LSTM (denoted as CNN-LSTM). Most importantly, the features learned from two streams interact in pairs, which increases the diversity of features and thereby further improves the performance. Finally, some comparisons with previous work are performed. The experimental results not only demonstrate the superior performance of the proposed method compared with the existing state-of-the-art methods, but also reveal the potential of DL-based approaches for AMC.

Automatic modulation classification (AMC) is becoming increasingly important in spectrum monitoring and cognitive radio. However, most existing modulation classification algorithms neglect the complementarities between different features and the importance of features fusion. To remedy these flaws, this paper presents a scheme of features fusion for AMC using convolutional neural network (CNN).The approach attempts to fuse different images and handcrafted features of signals to obtain more discriminating features. First, eight handcrafted features and different images features are both extracted. In the latter, signals are converted into two kinds of time-frequency images by smooth pseudo wigner-ville distribution (SPWVD) and Born-Jordan distribution (BJD), and a fine-tuned CNN model is utilized to extract image features. Second, the joint features are formed by combined of each of image and handcrafted features, and a multimodality fusion (MMF) model is applied to fuse the joint features to yield further improvement. Finally, simulation results reveal the superior performance of the proposed scheme. It is worth mentioning that the classification accuracy can reach 92.5\% with signal-to-noise ratio at -4dB.