Reliable recognition of unmanned aerial vehicle (UAV) communication signals is essential for low-altitude airspace safety and UAV monitoring. In practical electromagnetic environments, UAV signals exhibit complex time-frequency characteristics, and unknown signal types frequently appear, making open-set recognition necessary. This paper proposes a geometry-energy open-set recognition (GE-OSR) method for UAV signal identification. First, a time-frequency convolutional hybrid network is developed to learn multi-scale representations from raw UAV signals. Then, learnable class embeddings with a dual-constraint embedding loss are introduced to improve feature compactness and separability. In addition, a free-energy alignment loss is introduced to assign low energy to known signals and high energy to unknown ones, forming an adaptive rejection boundary. Experiments under different signal-to-noise ratios (SNRs) and openness levels show that GE-OSR provides stable performance. At 0 dB SNR under high openness, the method improves OSCR by about 2.95% over the recent S3R model and more than 6% over other baselines. These results show that GE-OSR is effective for practical UAV signal identification and unknown signal detection in complex low-altitude environments.

Reliable recognition of unmanned aerial vehicle (UAV) communication signals is important for low-altitude airspace safety and UAV monitoring systems. In real electromagnetic environments, UAV signals often show complex time-frequency patterns, and many unknown or new signal types can appear. These problems make open-set recognition necessary for practical UAV applications. In this paper, we propose a geometry-energy open-set recognition (GE-OSR) method designed for UAV signal monitoring. First, a time-frequency convolutional hybrid network is developed to learn multi-scale representations from raw UAV communication signals. Then, learnable class embeddings are added with a dual-constraint embedding loss to make the feature space more compact and separable. In addition, a free-energy alignment loss is introduced to guide the model to assign low energy to known UAV signals and high energy to unknown ones, which helps build an adaptive rejection boundary. Experiments under different SNRs and openness levels show that GE-OSR provides stable performance. At 0 dB SNR under high openness, the method improves OSCR by about 2.5% over the recent S3R model and more than 6% over other baselines. These results show that GE-OSR is effective for practical UAV signal identification and unknown signal detection in complex low-altitude environments.