Drone-based geo-localization is an image retrieval task which is the foundation of many drone-based multimedia applications, such as object detection, drone navigation and mapping. It is challenging due to the large visual appearance changes caused by viewpoint variation and time misalignment. Existing methods primarily focus on global representation embedding while disregarding the local features. We propose a CNN-based model containing a global and a local branch to extract features in these two perspectives and then features are subsequently aggregated by orthogonal fusion. We achieve competitive results on University-1652/160k datasets among the ViT-based state-of-the-art models. Experimental and qualitative results that validate the effectiveness of our solution are also shown.