Wenqi Gu

Mr. Wenqi Gu

orcid

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 September 2025 in Preprints.org
Verified
Preprint
Open access
PDF file
YZYingtao Zhang
Yingtao Zhang
WGWenqi Gu
Wenqi Gu
WHWen Hu
Wen Hu
JLJianguo Li
Jianguo Li
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
Transformers are the backbone of state-of-the-art systems across language, vision, and multimodal learning tasks, yet the relevance scale of their functional blocks (self-attention and feed-forward networks) is typically constant across inputs and depth. This static design neglects context-sensitive regulation of information flow through residual pathways. We introduce the \emph{contextual modulator}: a lightweight, input-aware mechanism that can scale the outputs of linear sublayers within a block or the entire block output at token- and channel-level granularity. The modulator is implemented via compact parametric functions and adds negligible parameter overhead. Building on this idea, we propose Transponder, which integrates contextual modulators throughout Transformer blocks to endow functional residual architectures with fine-grained, input-adaptive control. Transponder provides evident improvement over six other scaling or normalization methods across LLaMA backbones ranging from 60M to 250M parameters, yielding consistent perplexity reductions with $<1\%$ additional parameters. Analysis reveals depth-, module-, and token-specific scaling patterns, indicating that learned modulators act as input-adaptive regulators of residual information flow. Transponder provides a simple, general mechanism to augment Transformer-based models with context-sensitive modulators, providing robust and significant performance improvements without substantial architectural changes.

6 May 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
WGWenqi Gu
Wenqi Gu
YZYingtao Zhang
Yingtao Zhang
AMAlessandro Muscoloni
Alessandro Muscoloni
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
Logistic and logit functions play important roles in modern science, serving as foundational tools in various applications including artificial neural network (ANN). While there are functions that could produce distinct logistic and logit curves, no single, unified framework has been developed to generate both logistic and logit curves. We introduce a generalized logistic–logit function (CMG-GLLF) to fill this gap. CMG-GLLF provides four interpretable and trainable parameters that allow explicit control over: curve type and steepness, asymmetry, upper and lower limits of x- and y-axes. CMG-GLLF’s potential is explored in basic machine intelligence tasks. As a proof-of-concept on how this function can improve performance of deep learning, we propose a trainable input feature modulator (IFM) that consists in learning the parameters of the CMG-GLLF for each input layer node during backpropagation for multi-layer perceptron (MLP), which is a fundamental building block of many complex network architectures. Compared to various other learnable functions, across 3 different optimizers, CMG-GLLF allows superior MLP’s accuracy and stable training behavior on CIFAR-10 and CIFAR-100 image classification, but at the cost of increased computational time. Hence, we identified limitations to address in future studies, notably the need to derive an explicit mathematical expression for the logit phase, which could: (i) mitigate numerical instability in more complex architectures (e.g., CNNs) while reducing computational overhead, and (ii) enable a systematic evaluation of CMG as an activation function across all layers. Furthermore, CMG-GLLF adopted as data transformation function enhances the accuracy of affinity-graph-based neuron segmentation. CMG-GLLF combines in a unique framework the ability of logistic and logit function to modulate signals or variables, covering a full spectrum of attenuation or amplification transformations. CMG-GLLF is flexible and trainable, has potential to advance machine learning models, and can inspire further applications in other data analysis challenges in different domains of science.

29 April 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
WGWenqi Gu
Wenqi Gu
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
The recently proposed Cannistraci–Muscoloni–Gu Generalized Logistic–Logit Function (CMG-GLLF) introduced a flexible, trainable activation function capable of modulating input features in multilayer perceptrons (MLPs). However, its initial implicit approximation in the logit-phase caused computational overhead and numerical instability during training, limiting its application to more deep learning tasks and more complex network architectures. In this study, we derive a fully explicit and differentiable expression for the CMG-GLLF using a one-step Newton’s method approximation. We demonstrate that this new formulation resolves prior numerical instabilities and reduces computational overhead to match vanilla networks. When applied as activation function to input layers, it outperforms vanilla networks and linear modulators across MLPs, a simple CNN, and VGG-16 architectures on CIFAR-10/100, notably achieving superior performance on VGG-16 using a highly efficient "channel-wise" strategy that adds only 6 learnable parameters. Furthermore, we explore CMG-GLLF as a hidden-layer activation function. On image classification tasks, CMG-GLLF parameters naturally converge to approximate a ReLU-like shape, explaining its comparable performance. In contrast, on physics-informed neural networks (PINNs) solving 3 different physics process prediction tasks, CMG-GLLF learns activation functions highly different from standard Tanh activation function, achieving significantly better performance. These results demonstrate CMG-GLLF as a trainable activation function capable of explaining performance gain associated to learned node’s nonlinear functions in neural networks. Overall, this new formulation of CMG-GLLF provides powerful, scalable, and highly explainable node modulation for both classical and physics-informed neural networks.

21 August 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
KBKangyou Bao
Kangyou Bao
WGWenqi Gu
Wenqi Gu
JLjiaqing lyu
jiaqing lyu
XDXizheng Deng
Xizheng Deng
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
ANN-to-SNN conversion is an important approach for obtaining high-performance spiking neural networks (SNNs), yet current conversion methodologies predominantly focus on dense architectures, overlooking the struc- tural sparsity that characterizes biological neural systems. In this work, we propose a sparse ANN-to-SNN conversion framework that inherits learned sparse topology into SNNs. We instantiate this framework with Cannistraci-Hebb Training (CHT), a brain-inspired dynamic sparse training method. Across CNN experiments on CIFAR-10, CIFAR-100, and DVSGesture, and ViT experiments on ImageNet-1K, SNNs produced by our framework achieve near-dense performance while outperforming SNNs obtained from static pruning and sparse- training baselines in accuracy. Compared with dense SNN counterparts in the principal CIFAR and ImageNet comparisons, our framework also substantially reduces energy consumption. Beyond these accuracy-efficiency gains, the resulting sparse SNNs are characterized by biologically plausible structural properties, including meta- depth and scale-free organization. These results suggest that learned sparse topology is a useful design dimension for ANN-to-SNN conversion, enabling high-accuracy sparse SNNs with substantially reduced energy consumption.

30 September 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
TAThomas Adler
Thomas Adler
JCJunda Chen
Junda Chen
WGWenqi Gu
Wenqi Gu
ZLZiheng Liao
Ziheng Liao
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
Hyperbolic geometry explains many structural features of real-world networks, yet existing hyperbolic generative network models with community organization such as the nonuniform popularity-similarity optimization (nPSO) are restricted to unipartite graphs. Here we introduce a bipartite extension of the nPSO model called nBPSO, motivated by the prevalence of community organization in real-world bipartite networks and the emerging evidence of hyperbolic hierarchical connectivity in trained sparse neural networks. nBPSO provides explicit control over link density, degree distribution, average clustering, and community organization. We engineer the model to generate consecutive and concatenated bipartite layers, enabling initialization of dynamic sparse training multilayer perceptron neural networks with tunable hyperbolic organization for each layer. We find that task-agnostic (unaligned with the task) hyperbolic random network initialization degrades the performance of sparse neural networks compared with uniform random initialization. This degradation arises because the hyperbolic random initialization imposes an arbitrary hierarchical prior into the sparse topology. Dynamic sparse training may then need to optimize the weights and reorganize this arbitrary task-agnostic hierarchy into a task-driven one. By contrast, a uniform random initialization imposes at start less structural bias and may therefore be easier to reconfigure during training toward the hierarchical topology suited to the task. This work motivates the need for data-informed hyperbolic initialization methods which generate sparse topologies with a hierarchical prior closer to the task, facilitating the further learning of a task-oriented hierarchical organization during dynamic sparse training. In addition, nBPSO represents the first hyperbolic bipartite network generative model that allows the mesoscale community structure to be explicitly controlled through model parameters, making it a useful tool for broader complex systems applications where community organization is central to accurate and interpretable modeling.

30 September 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
WGWenqi Gu
Wenqi Gu
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
Physics-informed neural networks (PINNs) provide a data-efficient framework for solving partial differential equations, but improving their accuracy often requires larger multilayer perceptron (MLP) backbones, increasing parameter count and computational cost. We investigate whether learnable activation functions can improve PINN performance while preserving lightweight backbones and explainable nonlinearities. We adapt the Cannistraci-Muscoloni-Gu generalized logistic-logit function (CMG) to PINNs and introduce CMG-GELU, which replaces the tanh term in the GELU approximation with learnable CMG. To support stable PINN training, we reformulate CMG's implicit logit phase into an explicit differentiable approximation and adopt a layer-wise parameterization that adds only two trainable activation parameters per hidden layer. We compare CMG and CMG-GELU with nine representative activation and architecture baselines across 25 forward and inverse PINN tasks using five random seeds. Compared with representative PINN method studies, our 25-task suite covers substantially more tasks, enabling a broader cross-task evaluation. CMG-GELU delivers the strongest aggregate performance and the highest cross-task consistency among the 11 methods, achieving the best mean rank, rank standard deviation, and Robustness Performance Ranking Index (RPRI), while CMG ranks second by RPRI. Learned activation parameters reveal task- and layer-dependent activation modulation patterns, providing parameter-level explainability, while low-dimensional parameter-profile embeddings provide exploratory task-level evidence of local organization among tasks with related physical regimes. These results support CMG-based learnable activations as a lightweight and explainable approach to improving PINNs across heterogeneous problems.