Non-Intrusive Load Monitoring (NILM) is an effective energy consumption analysis technology, which just requires voltage and current signals on the user bus. This non-invasive monitoring approach can clarify the working state of multiple loads in the building with fewer sensing devices, thus reducing the cost of energy consumption monitoring. In this paper, an NILM method combining adaptive Recurrence Plot (RP) feature extraction and deep-learning-based image recognition is proposed. Firstly, the time-series signal of current is transformed into a threshold-free RP in phase space to obtain the image features. The Euclidean norm in threshold-free RP is scaled exponentially according to the voltage and current correlation to reflect the working characteristics of different loads adaptively. Afterwards, the obtained adaptive RP features can be mapped into images using the corresponding pixel value. In the load identification stage, an advanced computer vision deep network, Hierarchical Vision Transformer using Shifted Windows (Swin-Transformer), is applied to identify the adaptive RP images. The proposed solution is extensively verified by four real, measured load signal datasets, including industrial and household power situations, covering single-phase and three-phase electrical signals. The numerical results demonstrate that the proposed NILM method based on the adaptive RP can effectively improve the accuracy of load detection.