Aims: This study aims to investigate the diastolic left ventricular (LV) response to isometric handgrip exercise among healthy middle-aged men with high physical activity levels, versus matched sedentary individuals. Methods: Two groups of 10 men aged 41–51 years were studied. Men in the first group had high weekly self-reported physical activity levels (>3000 METs × min/week). In comparison, men in the second group reported low physical activity levels (<300 METs × min/week). An isometric handgrip exercise (IHE) stress echocardiography test was performed in all of them. Results: Both groups showed a similar and statistically significant increase in heart rate, systolic, diastolic, and mean arterial pressure following IHE. The group of active men under study did not show a statistically significant change in the ratio of early diastolic mitral valve inflow velocity to early diastolic lateral wall tissue velocity (E/e’ ratio) in response to IHE. Conversely, the inactive participants’ E/e’ ratio was higher at peak activity in the isometric handgrip exercise. Conclusions: Apparently, healthy middle-aged men with high levels of physical activity seem to have an improved lusitropic cardiac function compared to men with low levels of physical activity, as observed by the different diastolic LV responses induced by isometric handgrip exercise.