In non-cooperative communication systems such as radio spectrum resource regulation and modern electronic warfare, automatic modulation recognition is a key technology. Traditional modulation recognition methods mainly rely on manual feature extraction, decision theory, and recognition selection. The Deep Leaning (DL) algorithm automatically obtains signal features directly from massive data, and realizes feature extraction and recognition at the same time. However, most of the research on DL-AMR methods focuses on single input single output (SISO) systems, while there are few studies on DL-AMR methods in multiple-input, multiple-output (MIMO) systems, so the integration of deep learning models into modulation recognition of MIMO systems has attracted the attention of many researchers. The purpose of this paper is to provide a comprehensive review of modulation recognition methods for MIMOsystems based on DL. Firstly, the basic theory of MIMO and its derivative systems and modulation recognition is introduced in detail, then the traditional modulation recognition algorithms and deep learning-based modulation recognition algorithms of MIMO systems are introduced, and finally, on the basis of discussion and summary, the problems to be solved, the challenges and potential research directions are proposed.

Automatic modulation recognition is a critical step between signal detection and signal demodulation, and it is a critical technology for ensuring proper communication. Since the introduction of 5G technology, wireless communication systems have had massive data throughput, so combining deep learning technology with modulation recognition technology is now one of the most mainstream development directions in the field of communication. In order to effectively improve the robust recognition accuracy of modulation signals in low SNR condition, this paper proposes an end-to-end AMR model based on deep learning, called residual convolution long short memory Improved Transformer-Encoder deep neural network model (RLITNN). First, convolutional network and LSTM network extract the initial features from the communication signals of different modes. Second, the proposed Improved Transformer-Encoder module is used to capture the global and local focus of the extracted features. Then, these features are fused. Finally, the high quality features are further captured from the fused feature vector to enhance the feature representation ability of the model. Experiments on RML2016.10A and RML2016.10B datasets show that the proposed RLITNN has better feature learning ability and better recognition accuracy than other advanced (SOTA) techniques.

Radio spectrum resources are very limited and have become increasingly tight in recent years, and the exponential growth of various frequency-using devices has led to an increasingly complex and changeable electromagnetic environment. Wireless channel complexity and uncertainty have increased dramatically, and automated modulation recognition (AMR) performs poorly at low signal-to-noise ratios. It is proposed to use convolutional bidirectional long short-term memory deep neural networks (CNN-BiLSTM-DNNs) as a deep learning framework to extract features from single and mixed in-phase/orthogonal (I/Q) symbols in modulated data. The framework combines the capabilities of one- and two-dimensional convolution, a bidirectional long short-term memory network, and a deep neural network more efficiently, extracting characteristics from the perspective of time and space to enhance the accuracy of automatic modulation recognition. Modulation recognition experiments on the benchmark datasets RML2016.10b and RML2016.10a show that the average recognition accuracies of the proposed model from −20 dB to 18 dB are $64.76\%$ and $62.73\%$, respectively, and the improvement ranges of modulation recognition accuracy are 0.29−5.56% and 0.32−4.23% when the signal-to-noise ratio (SNR) is −10 dB to 4 dB, respectively. The CNN-BiLSTM-DNN model outperforms classical models such as MCLDNN, MCNet, CGDNet, ResNet, and IC-AMCNet in terms of modulation type recognition accuracy.