Distributed optical fiber vibration sensing system (DVS) base on phase-sensitive OTDR is widely used for its simple structure and high sensitivity. Signal recognition is crucial for DVS because it can help to classify the different types of vibration events. Deep learning provides accurate event classification and can automatically extract features according to sample distribution. However, almost all current methods focus on closed set recognition, which misclassifies unknown events into known categories thus reduces the recognition accuracy of sensing system. In this paper, we propose a novel open set event recognition model based on one-dimension residual learning convolution neural network (1-D RL-CNN) with OpenMax algorithm for DVS, which is capable of processing the signals of known and unknown categories. The experimental results show that the proposed recognition model improves the classification accuracy greatly compared with conventional 1-D CNN signal classification method. The overall open-set classification accuracy of 1-D RL-CNN with OpenMax is 91.19%, which is improved of 18.47% and 7.57% compared with 1-D CNN with SoftMax and 1-D CNN with OpenMax.