Traditional specific emitter identification (SEI) systems often suffer significant performance degradation when confronted with previously unseen signal sources, underscoring the critical need for accurate detection and rejection of novel-class instances. To address this limitation, we propose an Integrated Deep Feature Representation and Isolation Forest (IDFIF) method for identifying novel-class radiation emitters. IDFIF begins by employing a convolutional neural network (CNN) to extract embedding features from raw In-phase/Quadrature (IQ) signals, enhancing inter-class separability while suppressing intra-class variability. These deep features are then used to construct an unsupervised iForest that learns the statistical distribution of known classes, enabling the effective detection of anomalies via a threshold-based scoring mechanism. Experiments conducted on a real-world ADS-B dataset demonstrate that the proposed method achieves a novel-class detection accuracy of over 94%, significantly outperforming comparative methods. Furthermore, the method exhibits low sensitivity to known-class samples, thereby ensuring robustness and generalization under open-set conditions. The proposed IDFIF method is promising for deployment in challenging electromagnetic environments.