Shijun Zhang

Shijun Zhang

Auto-generated profile, based on publicly available information

Shijun Zhang

Department of Applied Mathematics, Hong Kong Polytechnic University, Hong Kong

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 September 2021 in Neural Networks
Verified
Journal Article
Closed access
ZSZuowei Shen
Zuowei Shen
HYHaizhao Yang
Haizhao Yang
SZShijun Zhang
Shijun Zhang

1 November 2019 in Neural Networks
Verified
Journal Article
Closed access
ZSZuowei Shen
Zuowei Shen
HYHaizhao Yang
Haizhao Yang
SZShijun Zhang
Shijun Zhang

26 March 2021 in Neural Computation
Verified
Journal Article
Closed access
ZSZuowei Shen
Zuowei Shen
HYHaizhao Yang
Haizhao Yang
SZShijun Zhang
Shijun Zhang
A new network with super-approximation power is introduced. This network is built with Floor (⌊x⌋) or ReLU (max{0,x}) activation function in each neuron; hence, we call such networks Floor-ReLU networks. For any hyperparameters N∈N+ and L∈N+, we show that Floor-ReLU networks with width max{d,5N+13} and depth 64dL+3 can uniformly approximate a Hölder function f on [0,1]d with an approximation error 3λdα/2N-αL, where α∈(0,1] and λ are the Hölder order and constant, respectively. More generally for an arbitrary continuous function f on [0,1]d with a modulus of continuity ωf(·), the constructive approximation rate is ωf(dN-L)+2ωf(d)N-L. As a consequence, this new class of networks overcomes the curse of dimensionality in approximation power when the variation of ωf(r) as r→0 is moderate (e.g., ωf(r)≲rα for Hölder continuous functions), since the major term to be considered in our approximation rate is essentially d times a function of N and L independent of d within the modulus of continuity.

18 September 2025 in SIAM Journal on Scientific Computing
Verified
Journal Article
Closed access
SZShijun Zhang
Shijun Zhang
HZHongkai Zhao
Hongkai Zhao
YZYimin Zhong
Yimin Zhong
HZHaomin Zhou
Haomin Zhou
In this work, we propose a balanced multicomponent and multilayer neural network (MMNN) structure to accurately and efficiently approximate functions with complex features in terms of both degrees of freedom and computational cost. The main idea is inspired by a multicomponent approach in which each component can be effectively approximated by a single-layer network, combined with a multilayer decomposition strategy to capture the complexity of the target function. Although MMNNs can be viewed as a simple modification of fully connected neural networks (FCNNs) or multilayer perceptrons (MLPs) by introducing balanced multicomponent structures, they achieve a significant reduction in training parameters, a much more efficient training process, and improved accuracy compared to FCNNs or MLPs. Extensive numerical experiments demonstrate the effectiveness of MMNNs in approximating highly oscillatory functions and their ability to automatically adapt to localized features. Our code and implementations are available at GitHub .

27 June 2025 in Information and Inference: A Journal of the IMA
Verified
Journal Article
Open access
SZShijun Zhang
Shijun Zhang
HZHongkai Zhao
Hongkai Zhao
YZYimin Zhong
Yimin Zhong
HZHaomin Zhou
Haomin Zhou
In this work, we present a comprehensive study combining mathematical and computational analysis to explain why a two-layer neural network struggles to handle high frequencies in both approximation and learning, especially when machine precision, numerical noise and computational cost are significant factors in practice. Specifically, we investigate the following fundamental computational issues: (1) the minimal numerical error achievable under finite precision, (2) the computational cost required to attain a given accuracy and (3) the stability of the method with respect to perturbations. The core of our analysis lies in the conditioning of the representation and its learning dynamics. Explicit answers to these questions are provided, along with supporting numerical evidence.

14 January 2023 in arXiv
Verified
Preprint
Open access
PDF file
ZSZuowei Shen
Zuowei Shen
HYHaizhao Yang
Haizhao Yang
SZShijun Zhang
Shijun Zhang
This paper proposes a new neural network architecture by introducing an additional dimension called height beyond width and depth. Neural network architectures with height, width, and depth as hyper-parameters are called three-dimensional architectures. It is shown that neural networks with three-dimensional architectures are significantly more expressive than the ones with two-dimensional architectures (those with only width and depth as hyper-parameters), e.g., standard fully connected networks. The new network architecture is constructed recursively via a nested structure, and hence we call a network with the new architecture nested network (NestNet). A NestNet of height $s$ is built with each hidden neuron activated by a NestNet of height $\le s-1$. When $s=1$, a NestNet degenerates to a standard network with a two-dimensional architecture. It is proved by construction that height-$s$ ReLU NestNets with $\mathcal{O}(n)$ parameters can approximate $1$-Lipschitz continuous functions on $[0,1]^d$ with an error $\mathcal{O}(n^{-(s+1)/d})$, while the optimal approximation error of standard ReLU networks with $\mathcal{O}(n)$ parameters is $\mathcal{O}(n^{-2/d})$. Furthermore, such a result is extended to generic continuous functions on $[0,1]^d$ with the approximation error characterized by the modulus of continuity. Finally, we use numerical experimentation to show the advantages of the super-approximation power of ReLU NestNets.