Auto License Plate Detection and Recognition (ALPDR) is required in many industrial fields, including visual surveillance systems and vehicle registration control. Even though this field has recently demonstrated high performance according to rapid advances in Deep Learning (DL) based technologies, there are still two problems that most related works have not yet solved. One is layout-dependent and the other is the problem of recognition accuracy degradation due to unconstrained environment conditions. In this paper, we present a more accurate, flexible and layout-independent method to improve LP detection and recognition accuracy under various outdoor conditions. For this, we proposed a new framework with lightweight and efficient anchor-free detection networks, employing the idea of CenterNet and attention-based recognition networks with residual deformable block suitable for LPR tasks. Various experiments demonstrate the performance of the proposed method, using famous LP benchmark datasets such as CCPD, AOLP and VBLPD. The proposed method outperformed the conventional LP detection and recognition methods. Additionally, Korean handicapped parking card (KHPC) data was tested to prove the usefulness of this method for marked character data different to license plates.