The recognition of point patterns in spatial vector data has important applications in geographic mapping and formation recognition. However, the application of traditional methods to spatial vector data faces two difficulties. Firstly, these data are low signal-to-noise ratio data in which the point patterns are mixed with a large number of normal point clusters; thus, it is difficult to recognize point patterns from these unstructured data using traditional clustering or machine learning methods. Secondly, the lack of edge connectivity relationships in spatial vector data directly hinders the application of graph models. Few studies have systematically solved the above difficulties. In this article, we propose a geometric feature attention scheme to overcome the above challenges. We also present an implementation of the scheme based on the graph method, termed GeoFAN, to extract and classify point patterns simultaneously in spatial vector data. Firstly, the raw data are transformed into a graph structure consisting of adjacency and attribute matrices. Secondly, a geometric feature attention module is proposed to enhance the feature representation of point patterns. Finally, the recognition results of all points are output via GeoFAN. The macro precision, recall, and $F_1$ score of five simulated point pattern types with different attributes and point numbers are 92.8%, 90.3%, and 91.5%, respectively, and GeoFAN is trained with simulated data to recognize real location-based point patterns successfully. The proposed GeoFAN showed superior performance and generalization ability in point pattern recognition.