Hybrid renewable energy systems are subject to extensive research around the world and different designs have found their way to the market and have been commercialized. These systems usually employ multiple components, both renewable and conventional, combined in a way to increase the system’s overall efficiency and resilience and to lower GHG emissions. In this paper, a hybrid renewable energy system was designed for residential use and its annual energy performance was investigated and optimized. The multi-module hybrid system consists of a Ground-Air Heat Exchanger (GAHX), Photovoltaic Thermal (PVT) panels and Air to Water Heat Pump (AWHP). The developed system’s annual performance was simulated in the TRaNsient SYStem (TRNSYS) environment and optimized using the General Algebraic Modelling System (GAMS) platform. Multi-objective non-linear optimization algorithms were developed and applied to define optimal system design and performance parameters while reducing cost and GHG emissions. The results revealed that the designed system was able to satisfy building thermal heating/cooling loads throughout the year. The ground source heat exchanger contributed 21.3% and 26.3% of the energy during heating and cooling seasons, respectively. The initial design was optimized in terms of key performance parameters and module sizes. The annual simulation analysis showed that the system was able to self-generate and meet nearly 29.4% of the total HVAC electricity needs, with the rest being supplied by the grid. The annual system module performance efficiencies were 13.4% for the PVT electric and 5.5% for the PVT thermal, with an AWHP COP of 4.0.