Accurate detection, quantitative assessment, and localization of delamination damage in composite materials remain major challenges in structural health monitoring (SHM). This paper proposes Adaptive Graph Transformer (AGFormer), a framework integrating a graph convolutional neural network with Transformer encoders, for size quantification, severity grading, and localization of circular delamination damage in carbon-fiber-reinforced polymer (CFRP) laminates. First, ten-dimensional edge features, including root-mean-square deviation, correlation coefficient, energy ratio, time-of-flight shift, and spectral variation, are extracted from differences between multipath Lamb-wave baseline and damaged-state signals. These features are aggregated into 42-dimensional enhanced node features to fuse multipath damage information at the sensor nodes. Second, a multimodal adjacency-matrix construction strategy combining sensor spatial distance with signal time–frequency distance is proposed to provide an adaptive topology for the graph convolutional network. On this basis, the AGFormer model employs three residual graph-convolution modules to extract local topological features, a multilayer perceptron for dimensional mapping, and two multi-head self-attention Transformer encoder layers to capture long-range dependencies among sensors. Attention-gated pooling and parallel output heads then simultaneously perform damage-radius regression and four-class severity classification. In addition, damage-centroid localization is achieved by combining multipath damage indices with the RAPID (Reconstruction Algorithm for Probabilistic Inspection of Damage) elliptical imaging algorithm. Experimental validation was conducted on a 450 mm × 450 mm six-ply CFRP laminate instrumented with a 12-piezoelectric-sensor network, under progressively increasing simulated delamination damage. The model was trained and evaluated using five-fold cross-validation grouped by damage size, while Gaussian-noise injection and random masking were introduced for data augmentation to improve generalization. The results demonstrate that the proposed method can effectively quantify and grade delamination damage of different sizes, achieving a mean absolute error of 4.38 mm, a root-mean-square error of 5.37 mm, and an R² of 0.920 for damage-radius regression, an accuracy of 81.3% for four-class severity classification, and a mean localization error of 6.50 mm, thereby offering a new technical approach to intelligent health monitoring of composite structures.