6 March 2023 in IEEE Transactions on Fuzzy Systems
Recent years have witnessed a drastic surge in graph representation learning, which usually produces low-dimensional and crisp representations from graph topology and high-dimensional node attributes. Nevertheless, a crisp representation of a node or graph actually conceals the uncertainty and interpretability of features. In citation networks, for example, the reference between two papers is always involved with fuzziness denoting the correlation degrees, that is, one connection may simultaneously belong to strong and weak references in different beliefs. The uncertainty in node connections and attributes inspires us to delve into fuzzy representations. This paper, for the first time, proposes an unsupervised fuzzy representation learning model for graphs and networks to improve their expressiveness by making crisp representations fuzzy. Specifically, we develop a fuzzy graph convolution neural network (FGCNN), which could aggregate high-level fuzzy features, leveraging fuzzy logic to fully excavate feature-level uncertainties, and finally generate fuzzy representations. The corresponding hierarchical model composed of multiple FGCNNs, called deep fuzzy graph convolution neural network (DFGCNN), is able to generate fuzzy node representations which are more expressive than crisp ones. Experimental results of multiple network analysis tasks validate that the proposed fuzzy representation models have strong competitiveness against the state-of-the-art baselines over several real-world datasets.
25 June 2024 in IEEE Transactions on Fuzzy Systems
Graph representation learning (GRL) focuses on abstracting critical information from raw graphs. Unfortunately, there always exist various kinds of uncertainties such as attribute noise and network topology corruption in raw graphs. Under the message passing mechanism, these uncertainties are likely to spread throughout the whole graph. Matters like these would induce deep graph models into producing uncertain representations and restrict representation expressiveness. Considering this, we propose a pioneering framework to defend graph uncertainties by improving the robustness and capability of graph neural networks (GNNs). In our framework, we consider that weights and biases are all fuzzy numbers, thus generating representations to assimilate graph uncertainties which are finally released by defuzzification. To describe the process of the framework, in this paper, a graph convolutional network (GCN) is employed to construct a robust graph model, called FuzzyGCN. To verify the effectiveness of FuzzyGCN, it is trained in both supervised and unsupervised ways. In the supervised setting, we find that FuzzyGCN has stronger power and is more immune to data uncertainties when compared with various classical and robust GNNs. In the unsupervised setting, FuzzyGCN surpasses many state-of-the-art models in node classification and community detection over several real-world datasets.
1 October 2025 in IEEE Transactions on Knowledge and Data Engineering
30 December 2022 in IEEE Transactions on Computational Social Systems
Information aggregation and propagation over networks via graph neural networks (GNNs) plays an important role in node or graph representation learning, which currently depend on the calculation with a fixed adjacency matrix, facing over-smoothing problem, and difficulty to stack multiple layers for high-level representations. In contrast, Transformer calculates an importance score for each node to learn its embedding via the attention mechanism and has achieved great successes in many natural language processing (NLP) and computer vision (CV) tasks. However, Transformer is inflexible to extend to graphs, as its input and output must have the same dimension. It will also become intractable to allocate attention over a large-scale graph due to distractions. Moreover, most graph Transformers are trained in supervised ways, which consume additional resources to annotate samples with potentially wrong labels and have limited generalization of representations. Therefore, this article attempts to build a new Sparse Graph Transformer with Contrastive learning for graph representation learning, called SGTC. Specifically, we first employ centrality measures to remove the redundant topological information from input graph according to the influences of nodes and edges, then disturb the pruned graph to get two different augmentation views, and learn node representations in a contrastive manner. Besides, a novel sparse attention mechanism is also proposed to capture structural features of graphs, which effectively save memory and training time. SGTC can produce low-dimensional and high-order node representations, which have better generalization for multiple tasks. The proposed model is evaluated on three downstream tasks over six networks, and experimental results confirm its superior performance against the state-of-the-art baselines.
25 January 2023 in IEEE Transactions on Network Science and Engineering
Contrastive learning has been widely used in graph representation learning, which extracts node or graph representations by contrasting positive and negative node pairs. It requires node representations (embeddings) to reflect their correlations in topology, increasing the similarities between an anchor node and its positive nodes, or reducing the similarities with its negative nodes in embedding space. However, most existing contrastive models measure similarities through a fixed metric that equally scores all sample pairs in a specific feature space, but ignores the varieties of node attributes and network topologies. Moreover, these fixed metrics are always defined explicitly and manually, which makes them unsuitable for applying to all graphs and networks. To solve these problems, we propose a novel graph representation learning model with an adaptive metric, called GRAM, which produces appropriate similarity scores of node pairs according to the different significance of each dimension in their embedding vectors and adaptive metrics based on data distribution. With these scores, it is better to train a graph encoder and obtain representative embeddings. Experimental results show that GRAM has strong competitiveness in multiple tasks.
16 December 2024 in IEEE Transactions on Cognitive and Developmental Systems
The goal of multi-document summarization (MDS) is to generate a comprehensive and concise summary from multiple documents, which should not only be grammatically correct but also semantically contains the refined content of the overall texts. Existing summarizers based on sequential pre-trained large language models often cognize documents as linear sequences, which overlook the hierarchical structure correlations of sentences and paragraphs within or between documents. Additionally, those models also have limitations in handling long text input. To alleviate these two problems, a multi-document summarization model is proposed, with a heterogeneous graph of sentences, paragraphs and documents, called HeterMDS, to uncover deep semantic meanings and local-global context within documents. By integrating large language model and graph encoder with bootstrapped graph latents, the proposed HeterMDS can learn a semantically rich document representation and generate a coherent, concise and fact-consistent summary. It can be flexibly applied to current pre-trained language models, effectively improving their performance in MDS. Extensive experiment results can verify the effectiveness of the proposed HeterMDS and its contained modules, and demonstrate its competitiveness against the state-of-the-art models.