Land Surface Temperature (LST) is a key variable used across various applications, including irrigation monitoring, vegetation health assessment and urban heat island studies. While satellites offer moderate-resolution LST data, unmanned aerial vehicles (UAVs) provide high-resolution thermal infrared measurements. However, the continuous and rapid variation in LST makes the production of orthomosaics from UAV-based image collections challenging. Understanding the environmental and meteorological factors that amplify this variation is necessary to select the most suitable conditions for collecting UAV-based thermal data. Here, we capture variations in LST while hovering for 15–20 min over diverse surfaces, covering sand, water, grass, and an olive tree orchard. The impact of different flying heights and times of the day was examined, with all collected thermal data evaluated against calibrated field-based Apogee SI-111 sensors. The evaluation showed a significant error in UAV-based data associated with wind speed, which increased the bias from −1.02 to 3.86 °C for 0.8 to 8.5 m/s winds, respectively. Different surfaces, albeit under varying ambient conditions, showed temperature variations ranging from 1.4 to 6 °C during the flights. The temperature variations observed while hovering were linked to solar radiation, specifically radiation fluctuations occurring after sunrise and before sunset. Irrigation and atmospheric conditions (i.e., thin clouds) also contributed to observed temperature variations. This research offers valuable insights into LST variations during standard 15–20 min UAV flights under diverse environmental conditions. Understanding these factors is essential for developing correction procedures and considering data inconsistencies when processing and interpreting UAV-based thermal infrared data and derived orthomosaics.