The non-terrestrial network (NTN) is a network that uses radio frequency (RF) resources mounted on satellites and includes satellite-based communications networks, high altitude platform systems (HAPS), and air-to-ground networks. The fifth generation (5G) and NTN may be crucial in utilizing communication infrastructure to provide 5G services in the future, anytime and anywhere. Based on the outcome of the Rel-16 study, the 3rd generation partnership project (3GPP) decided to start a work item on an NTN in 5G new radio (NR) Rel-17, and the focus of the study was on mobility management procedures, due to the movements of NTN platforms; especially, low earth orbit (LEO) satellites. Handover enhancements were discussed to tackle the frequent handover due to the fast satellite movement. Therefore, the major problem of handover in LEO satellite systems was the signaling storm created by handing over all user equipment (UE) in a cell to a new cell because when the UE crosses the boundary between the neighboring cell of a satellite, an intra-satellite or cell handover occurs; thus, all users in a cell are expected to experience a change of cell due to handover every few seconds. In addition, UE location is not easy to define due to moving cell/beam situations. In this study, we propose machine learning-based solutions for handover decisions in non-terrestrial networks for cell handovers or intra-satellite handovers to reduce signaling storms during handovers where the handover requests will be executed by clustered users. First, the dataset was generated by the simulator that simulates communication between users and satellites. Second, we preprocessed the data, and also used the feature creation technique to create the distance feature using the Haversine formula, and then applied clustering and classification algorithms. The experimental results show that the distance between a user and its cell center is an important parameter for handover decisions in NTN, and the random forest outperforms all models with a higher accuracy of 99% along with a better F1-score of 0.9961.

This study proposes a reinforcement learning-based link selection method for Low Earth Orbit satellite networks, aiming to reduce handover frequency while extending link duration under highly dynamic orbital environments. The proposed approach relies solely on basic satellite positional information, namely latitude, longitude, and altitude, to construct compact state representations without requiring complex sensing or prediction mechanisms. Using relative satellite and terminal geometry, each state is represented as a vector consisting of azimuth, elevation, range, and direction difference. To validate the feasibility of policy learning under realistic conditions, a total of 871,105 orbit based data samples were generated through simulations of 300 LEO satellite orbits. The reinforcement learning environment was implemented using the OpenAI Gym framework, in which an agent selects an optimal communication target from a prefiltered set of candidate satellites at each time step. Three reinforcement learning algorithms, namely SARSA, Q-Learning, and Deep Q-Network, were evaluated under identical experimental conditions. Performance was assessed in terms of smoothed total reward per episode, average handover count, and average link duration. The results show that the Deep Q-Network-based approach achieves approximately 77.4% fewer handovers than SARSA and 49.9% fewer than Q-Learning, while providing the longest average link duration. These findings demonstrate that effective handover control can be achieved using lightweight state information and indicate the potential of deep reinforcement learning for future LEO satellite communication systems.