One of the main capabilities a mobile robot must demonstrate is the ability to explore its environment. The core challenge in exploration lies in planning the route to fully cover the environment. Despite recent advances, this problem remains unsolved. This study proposes an approach to address the coverage path planning problem, where the mobile robot is tasked with exploring and completely covering a terrain using a deep reinforcement learning framework. The environment is divided into cells, with obstacles designated as prohibited areas. The robot is trained using two state-of-the-art reinforcement learning algorithms based on actor–critic methods: Advantage Actor–Critic (A2C) and Proximal Policy Optimization (PPO). By defining a set of observations, states, and a reward function tailored to characteristics of the environment and the desired behavior of the robot, the training process is conducted, resulting in optimized policies for each algorithm. Then, these policies are evaluated to determine the most effective approach to accomplish the proposed task. Our findings demonstrate that actor–critic methods can produce policies capable of guiding a robot to efficiently explore and cover new environments.