With the development of deep learning, several graph neural network (GNN)-based approaches have been utilized for text classification. However, GNNs encounter challenges when capturing contextual text information within a document sequence. To address this, a novel text classification model, RB-GAT, is proposed by combining RoBERTa-BiGRU embedding and a multi-head Graph ATtention Network (GAT). First, the pre-trained RoBERTa model is exploited to learn word and text embeddings in different contexts. Second, the Bidirectional Gated Recurrent Unit (BiGRU) is employed to capture long-term dependencies and bidirectional sentence information from the text context. Next, the multi-head graph attention network is applied to analyze this information, which serves as a node feature for the document. Finally, the classification results are generated through a Softmax layer. Experimental results on five benchmark datasets demonstrate that our method can achieve an accuracy of 71.48%, 98.45%, 80.32%, 90.84%, and 95.67% on Ohsumed, R8, MR, 20NG and R52, respectively, which is superior to the existing nine text classification approaches.

With the development of deep learning, several Graph Neural Networks (GNN)-based approaches have been utilized for text classification. However, GNNs encounter challenges in capturing contextual text information within a document sequence. To address this, a novel text classification model RB-GAT is proposed by combining RoBERTa-BiGRU embedding and a multi-head Graph ATtention Network (GAT). First, the pre-trained RoBERTa model is exploited to learn word and text embeddings in different contexts. Second, the Bidirectional Gated Recurrent Unit (BiGRU) is employed to capture long-term dependencies and bidirectional sentence information from the text context. Next, the multi-head graph attention network is applied to analyze this information, which serves as a node feature for the document. Finally, the classification results are generated through a Softmax layer. Experimental results on three benchmark datasets demonstrate that our method can achieve an accuracy of 71.48%, 98.45%, and 80.32% on Ohsumed, R8, and MR, which is superior to the existing nine text classification approaches.

Network embedding has attracted considerable attention in recent years. It represents nodes in a network into a low-dimensional vector space while keeping the properties of the network. Some methods (e.g. ComE, MNMF, and CARE) have been proposed to preserve the community property in network embedding, and they have obtained good results in some downstream network analysis tasks. However, there still exists a significant challenge because nodes may lose important structural information following embedding. To address this problem, we propose a community structure enhancement framework for network embedding, based on edge reweighting. Through edge reweighting, the weight of intra-community edges is increased while the weight of inter-community edges is decreased. Therefore, after embedding, nodes in the same community are closer to each other than nodes in different communities in the embedding space. We apply the edge reweighting as a preprocessing stage in network embedding, and construct an enhanced network by incorporating enhanced community structures into the original network. By doing this, the embedded vectors from the enhanced network can better perform all downstream network analysis tasks. Extensive experiments are conducted on two network analysis tasks (community detection and node classification) with synthetic and real-world datasets. The results show that our method outperforms state-of-the-art network embedding methods.

With the development of Graph Neural Network (GNN), a lot of GNN-based methods have been proposed in text classification. However, GNNs are difficult to capture the word relationship of the context in the document sequence. In addition, these models are common to use one-hot encoding to initialize the node features, which have high dimensions and sparse features. Therefore, it is difficult to represent the correlation between texts. To effectively learn the semantic features of contexts and the association between words, we propose a text classification method based on the BERT-BiLSTM word embedding model and the Graph Convolutional Network (GCN), which is named BBG. The BERT-BiLSTM can learn word embedding representations in different contexts and the semantic relationship of text contexts. The learned representation vectors are provided into GCN as the features of document nodes. The feature representations of the training data and unlabeled test data are learned by performing the label propagation in GCN. Finally, the classification results are output by utilizing the Softmax layer. Experiments on four benchmark datasets show that our method has better performances than existing text classification methods.

Network embedding has attracted a lot of attention in different fields recently. It represents nodes in a network into a low-dimensional and dense space while preserving the structural properties of the network. Some methods (e.g. motif2Vec, RUM, and MODEL) have been proposed to preserve the higher-order structures, i.e., motifs in embedding space, and they have obtained better results in some downstream network analysis tasks. However, there still exists a significant challenge because original motifs may include redundant noise edges, and embedding entire motifs into embedding space may adversely affect the performance in downstream tasks. To overcome this problem, we propose a motifs enhancement framework for network embedding, based on edge reweighting. Through edge reweighting, the weight of redundant noise edges between motifs is decreased. Therefore, the effect of redundant noise edges will be reduced in the embedding space. We apply the edge reweighting as a preprocessing phase in network embedding, and construct the motifs enhanced network by incorporating enhanced motifs structures with the original network. By doing this, the embedding vectors from the motifs enhanced network can achieve better performance in downstream network analysis tasks. Extensive experiments are performed on two network analysis tasks (community detection and node classification) with synthetic and real-world datasets. The results show that our framework outperforms state-of-the-art network embedding methods.