This letter proposes an ensemble neural network (Ensem-NN) for skeleton-based action recognition. The Ensem-NN is introduced based on the idea of ensemble learning, “two heads are better than one”. According to the property of skeleton sequences, we design one-dimensional (1D) convolution neural network (CNN) with residual structure as Base-Net. From entirety to local, from focus to motion, we designed four different subnets based on the Base-Net to extract diverse features. The first subnet is a Two-stream Entirety Net, which performs on the entirety skeleton and explores both temporal and spatial features. The second is a Body-part Net, which can extract fine-grained spatial and temporal features. The third is an Attention Net, in which a channel-wised attention mechanism can learn important frames and feature channels. Frame-difference Net, as the fourth subnet, aims at exploring motion features. Finally, the four subnets are fused as one ensemble network. Experimental results show that the proposed Ensem-NN performs better than state-of-the-art methods on three widely used datasets.