Lots of recent deep learning based intelligent fault diagnosis methods of planetary gearbox have achieved satisfactory accuracy with balanced training samples. Nevertheless, the fault samples are generally far less than healthy samples in practical engineering, and the collected data samples usually contain lots of noise, making it difficult to achieve accurate fault diagnosis. In order to solve these problems, this paper proposes a new method called novel multi-scale dilated convolutional neural network with long short-term memory (CNN-LSTM). Firstly, a novel multi-scale dilated CNN is constructed using new dilated strategy to enrich the coverage of the fields of view and avoid the loss of original information, which could adequately mine the distinguishing features of small samples. Secondly, an adaptive weight unit combined with LSTM is designed to fuse the distinguishing features and improve their robustness to noise. Finally, to pay more attention to the small samples and easily confused samples, a new-type loss function called enhanced cross entropy is developed. The test and analysis of the planetary gearbox data sets prove that the proposed method shows better diagnosis performance than other comparison methods using unbalanced training samples.