The demand for low-latency computing from the Internet of Things (IoT) and emerging applications challenges traditional cloud computing. Mobile Edge Computing (MEC) offers a solution by deploying resources at the network edge, yet terrestrial deployments face limitations. Unmanned Aerial Vehicles (UAVs), leveraging their high mobility and flexibility, provide dynamic computation offloading for User Equipments (UEs), especially in areas with poor infrastructure or network congestion. However, UAV-assisted MEC confronts significant challenges, including time-varying wireless channels and the inherent energy constraints of UAVs. We put forward the Lyapunov-based Deep Deterministic Policy Gradient (LyDDPG), a novel computation offloading algorithm. This algorithm innovatively integrates Lyapunov optimization with the Deep Deterministic Policy Gradient (DDPG) method. Lyapunov optimization transforms the long-term, stochastic energy minimization problem into a series of tractable, per-timeslot deterministic subproblems. Subsequently, DDPG is utilized to solve these subproblems by learning a model-free policy through environmental interaction. This policy maps system states to optimal continuous offloading and resource allocation decisions, aiming to minimize the Lyapunov-derived “drift-plus-penalty” term. The simulation outcomes indicate that, compared to several baseline and leading algorithms, the proposed LyDDPG algorithm reduces the total system energy consumption by at least 16% while simultaneously maintaining low task latency and ensuring system stability.