In fire smoke detection, interference aerosols frequently cause false alarms for photoelectric smoke detectors. Conventional closed-set classification models trained on limited samples cannot reliably handle unknown aerosol types, which further leads to false and missed alarms. To address this issue, we propose FireOSR-Net, an open-set recognition network for fire smoke identification using the signals of a four-channel optical scattering detector, which integrates three modules. First, Conv1d-LSTM is used to extract features from dual-wavelength, dual-angle scattered signals. Next, the out-of-distribution detection module constructs an open boundary between known and unknown aerosols and generates pseudo-unknown features to enhance the detection capability for out-of-domain samples. Finally, the distance of all scattered intensity signals to both the fire and the non-fire domain is calculated, and fire and non-fire proximity scores are output to assess their risk propensity. Experiments are performed on eight-class aerosol data acquired with our self-developed four-channel detector, including six known and two unknown test categories. It achieves a peak F1-score of 0.95 on known samples, while unknown smoldering-wire smoke and oil fumes reach 96.29% fire proximity and 89.68% non-fire proximity correspondingly. Such results demonstrate FireOSR-Net’s capacity for aerosol classification, out-of-domain detection and risk evaluation towards interpretable low-false-alarm fire sensing.