Learning powerful discriminative features is the key for remote sensing scene classification. Most existing approaches based on Convolutional Neural Network (CNN) have achieved great results. However, they mainly focus on global-based visual features while ignoring object-based location features which is important for large-scale scene classification. There are a large number of scene-related ground objects in remote sensing images, as well as Graph convolutional network (GCN) has the potential to capture the dependencies among objects. This paper introduces a novel two-stream architecture that combines global-based visual features and object-based location features, so as to improve the feature representation capability. Firstly, we extract appearance visual features from whole scene image based on CNN. Secondly, we detect ground objects and construct a graph to learn the spatial location features based on GCN. As a result, the network can jointly capture appearance visual information and spatial location information. To the best of our knowledge, we are the first to investigate the dependencies among objects in remote sensing scene classification task. Extensive experiments on two datasets show that our framework improves the discriminative ability of features and achieves competitive accuracy against other state-of-the-art approaches.