The Service Function Chain (SFC) provisioning problem in a 5G/B5G network corresponds to the routing of service requests through an ordered list of virtualized network functions (VNFs) while respecting network/computing resource, and delay/bandwidth constraints. This study proposes a solution to this problem through the use of Deep Reinforcement Learning (Deep-RL) and Graph Neural Networks (GNNs). The latter ones are first explored with Graph Convolutional Network (GCN) and then with Graph Attention Networks (GATs) which leverage attention mechanisms for feature learning on graphs. Experiments are conducted on large graphs with up to 200 nodes. GNN (both GCN and GAT) solutions compare very favorably to a similar agent employing a fully-connected network at its core, as they require less data to learn better and faster the topology, with GAT improving over GCN.