Agrivoltaic systems, which integrate photovoltaic panels on agricultural surfaces, offer protection against heat-waves, droughts, and frosts by altering the microclimate in the vicinity of the panels. Although photosynthesis may be reduced due to partial sky obstruction by photovoltaic panels, a trade-off emerges when these panels help alleviate plant stress, ultimately increasing agricultural yield. Therefore, in regions prone to drought, frost, or strong wind, optimizing agrivoltaic system design becomes crucial for reducing weather-related risks. This requires a thorough understanding of how each geometric parameter of the photovoltaic panels influences the microclimate. Using a soil-plant-atmosphere continuum model implemented in the computational fluid dynamics solver code_saturne, the effects of several agrivoltaic power plant configurations on microclimate and their resulting impacts on plant thermal conditions are simulated. This model coupling is validated on plant temperature measurements made at an experimental agrivoltaic power plant in France, with an error lower than 3°C. Then, it is applied in three scenarios: (1) white frost conditions, (2) drought conditions, and (3) high speed wind conditions. 49 simulations are conducted, revealing differences in spatially-averaged plant temperature often exceeding 5°C among the evaluated agrivoltaic systems. All in all, these results highlight the importance to tailor agrivoltaic designs to the predominant stresses plants are likely to face, in order to optimize plant protection and, consequently, agricultural yield.