Autonomous driving is a complex problem that requires intelligent decision making, and it has recently garnered significant interest due to its potential advantages in convenience and safety. In autonomous driving, conventional path planning to reach a destination is a time-consuming challenge. Therefore, learning-based approaches have been successfully applied to the controller or decision-making aspects of autonomous driving. However, these methods often lack explainability, as passengers cannot discern where the vehicle is headed. Additionally, most experiments primarily focus on highway scenarios, which do not effectively represent road curvature. To address these issues, we propose a reinforcement-learning-based trajectory learning in the Frenet frame (RLTF), which involves learning trajectories in the Frenet frame. Learning trajectories enable the consideration of future states and enhance explainability. We demonstrate that RLTF achieves a 100% success rate in the simulation environment, considering future states on curvy roads with continuous obstacles while overcoming issues associated with the Frenet frame.