Jialin Zhao

Jialin Zhao


Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 October 2025 in Preprints.org
Verified
Preprint
Open access
PDF file
WWWenjing Wu
Wenjing Wu
YZYingtao Zhang
Yingtao Zhang
JZJialin Zhao
Jialin Zhao
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
With the rapid development of large language models (LLMs), identifying efficient strategies for training such large-scale systems has become increasingly critical. Although LLMs have achieved remarkable success across diverse applications, the necessity of maintaining full dense matrices during pre-training has been questioned, giving rise to parameter-efficient sparse pre-training methods which retains parameter-efficiency in both training and inference. These methods can be further divided into connectivity sparse training and spectral sparse training, with dynamic connectivity sparse training and low-rank factorization emerging as representative approaches for the two branches. However, a unified framework that effectively combines the strengths of both has yet to be established. In this work, we observe that the cancellation effect between the sparse and low-rank branches may limit the expressivity of the model, manifesting as output conflicts when the two components are combined. To address this issue, we propose a novel scheme that integrates dynamic sparse training with low-rank training, introducing a simple yet effective alignment loss to mitigate the disagreement between the two branches and promote better collaboration. We validate this scheme by combining a representative dynamic sparse training method, CHTs, with low-rank training, resulting in a new parameter-efficient training approach termed CHTsL. The method is evaluated on LLaMA60M and LLaMA130M using the OpenWebText and C4 datasets, where only 10\%, 20\%, and 30\% of the parameters are preserved compared to dense training. Experimental results demonstrate that our proposed scheme effectively alleviates the cancellation effect and improves training stability and performance compared to the naive combination of sparse and low-rank components. Also, the new scheme enables CHTsL to consistently outperform other parameter-efficient sparse training methods under the same parameter budget, achieving performance most close to dense training.

4 January 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
JZJialin Zhao
Jialin Zhao
AMAlessandro Muscoloni
Alessandro Muscoloni
UMUmberto Michieli
Umberto Michieli
YZYingtao Zhang
Yingtao Zhang
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
Many complex networks have partially observed or evolving connectivity, making link prediction a fundamental task. Topological link prediction infers missing links using only network topology, with applications in social, biological, and technological systems. The Cannistraci-Hebb (CH) theory provides a topological formulation of Hebbian learning, grounded on two pillars: (1) the minimization of external links within local communities, and (2) the path-based definition of local communities that capture homophilic (similarity-driven) interactions via paths of length 2 and synergetic (diversitydriven) interactions via paths of length 3. Building on this, we introduce the Cannistraci-Hebb Adaptive (CHA) network automata, an adaptive learning machine that automatically selects the optimal CH rule and path length to model each network. CHA unifies theoretical interpretability and data-driven adaptivity, bridging physics-inspired network science and machine intelligence. Across 1,269 networks from 14 domains, CHA consistently surpasses state-of-the-art methods—including SPM, SBM, graph embedding methods, and message-passing graph neural networks—while revealing the mechanistic principles governing link formation. Our code is available at https://github.com/biomedical-cybernetics/Cannistraci_Hebb_network_automata.

2 April 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
YWYue Wu
Yue Wu
JZJialin Zhao
Jialin Zhao
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
With the advent of the neural network era, traditional machine learning methods have increasingly been overshadowed. Nevertheless, continuing research on the role of geometry for learning in data science is crucial to envision and understand new principles behind the design of efficient machine learning. Linear classifiers are favored in certain tasks due to their reduced susceptibility to overfitting and their ability to provide interpretable decision boundaries. However, achieving both scalability and high predictive performance in linear classification remains a persistent challenge. Here, we propose a theoretical framework named geometric discriminant analysis (GDA). GDA includes the family of linear classifiers that can be expressed as a function of a centroid discriminant basis (CDB0) - the connection line between two centroids - adjusted by geometric corrections under different constraints. We demonstrate that linear discriminant analysis (LDA) is a subcase of the GDA theoretical framework, and we show its convergence to CDB0 under certain conditions. Then, based on the GDA framework, we propose an efficient linear classifier named centroid discriminant analysis (CDA) which is defined as a special case of GDA under a 2D plane geometric constraint. CDA training is initialized starting from CDB0 and involves the iterative calculation of new adjusted centroid discriminant lines whose optimal rotations on the associated 2D planes are searched via Bayesian optimization. CDA has good scalability (quadratic time complexity) which is lower than LDA and support vector machine (SVM) (cubic complexity). Results on 27 real datasets across classification tasks of standard images, medical images and chemical properties, offer empirical evidence that CDA outperforms other linear methods such as LDA, SVM and logistic regression (LR) in terms of scalability, performance and stability. Furthermore, we show that linear CDA can be generalized to nonlinear CDA via kernel method, demonstrating improvements on the linear version with tests on three challenging datasets of images and chemical data. GDA's general validity as a new theoretical framework may inspire the design of new classifiers under the definition of different geometric constraints, paving the way towards a deeper understanding of the role of geometry in learning from data.

15 February 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
WWWenjing Wu
Wenjing Wu
YZYingtao Zhang
Yingtao Zhang
JZJialin Zhao
Jialin Zhao
Carlo Vittorio CannistraciCarlo Vittorio Cannistraci
Carlo Vittorio Cannistraci
In recommendation systems, representing user-item interactions as a bipartite network is a fundamental approach that provides a structured way to model relationships between users and items, allowing for efficient predictions via network science. Collaborative filtering is one of the most widely used and actively researched techniques for recommendation systems, its rationale is to predict user preferences based on shared patterns in user interactions, and vice versa. Memory-based collaborative filtering relies on directly analyzing user-item interactions to provide recommendations using similarity measures, and differs from model-based collaborative filtering which builds a predictive model using machine learning techniques such as neural networks. With the rise of machine learning, memory-based collaborative filtering has often been overshadowed by model-based approaches. However, the recent success of SSCF, a newly proposed memory-based method, has renewed interest in the potential of memory-based approaches. In this paper, we propose Network Shape Automata (NSA), a memory-based collaborative filtering method grounded in the connectivity shape of the bipartite network topology. NSA leverages the Cannistraci-Hebb theory proposed in network science to define brain-inspired network automata, using this paradigm as the foundation for its similarity measure. We evaluate NSA against a range of advanced collaborative filtering methods, both memory-based and model-based, across 16 bipartite network datasets spanning complex systems domains such as social networks and biological networks. Results show that NSA consistently achieves strong performance across diverse datasets and evaluation metrics, ranking most often first on average. Notably, NSA demonstrates strong robustness to network sparsity, while preserving the simplicity, interpretability, and training-free nature of memory-based methods. As a pioneering effort to bridge link prediction and recommendation tasks, NSA not only highlights the untapped potential of memory-based collaborative filtering but also demonstrates the effectiveness of the Cannistraci-Hebb theory in modeling network evolution within recommendation systems.

2 October 2025 in arXiv
Verified
Preprint
Open access
PDF file
JZJialin Zhao
Jialin Zhao
Attention is a core operation in large language models (LLMs) and vision-language models (VLMs). We present BD Attention (BDA), the first lossless algorithmic reformulation of attention. BDA is enabled by a simple matrix identity from Basis Decomposition (BD), which restructures multi-head projections into a compact form while preserving exact outputs. Unlike I/O-aware system optimizations such as FlashAttention, BDA provides a mathematically guaranteed acceleration that is architecture-agnostic. On DeepSeek-V2-Lite (16B, FP16), BDA requires only 4s of offline preparation with no retraining required and, on modern GPUs, achieves 32% faster key/value projections and 25% smaller weights, while increasing end-to-end perplexity (PPL) by just 0.02% (FP16) or 0.0004% (FP32), a negligible effect on model performance. These results position BDA as the first theoretically exact method for lossless attention acceleration that is complementary to existing engineering-level optimizations. Our code is available at https://github.com/abcbdf/basis-decomposition-official.