Haiping Huang

Haiping Huang

Auto-generated profile, based on publicly available information

Haiping Huang

School of Physics PMI Lab Sun Yat-sen University 510275 Guangzhou People's Republic of China and

Research Focus & Metrics

Academic Career Length

35Years (1992-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
18 August 2016 in BMC Neuroscience
Verified
Journal Article
Open access
TEthe EyeWirers
the EyeWirers
TSTatyana O. Sharpee
Tatyana O. Sharpee
ADAlain Destexhe
Alain Destexhe
MKMitsuo Kawato
Mitsuo Kawato
VSVladislav Sekulić
Vladislav Sekulić
KCKarishma Chhabria
Karishma Chhabria
A1 Functional advantages of cell-type heterogeneity in neural circuits

17 November 2014 in Physical Review E
Verified
Journal Article
Closed access
PDF file
HHHaiping Huang
Haiping Huang
YKYoshiyuki Kabashima
Yoshiyuki Kabashima
Through supervised learning in a binary perceptron one is able to classify an extensive number of random patterns by a proper assignment of binary synaptic weights. However, to find such assignments in practice is quite a nontrivial task. The relation between the weight space structure and the algorithmic hardness has not yet been fully understood. To this end, we analytically derive the Franz-Parisi potential for the binary perceptron problem by starting from an equilibrium solution of weights and exploring the weight space structure around it. Our result reveals the geometrical organization of the weight space; the weight space is composed of isolated solutions, rather than clusters of exponentially many close-by solutions. The pointlike clusters far apart from each other in the weight space explain the previously observed glassy behavior of stochastic local search heuristics.

23 August 2013 in Journal of Physics A: Mathematical and Theoretical
Verified
Journal Article
Closed access
HHHaiping Huang
Haiping Huang
KWK Y Michael Wong
K Y Michael Wong
YKYoshiyuki Kabashima
Yoshiyuki Kabashima

1 May 2017 in Journal of Statistical Mechanics: Theory and Experiment
Verified
Journal Article
Closed access
PDF file
HHHaiping Huang
Haiping Huang
Revealing hidden features in unlabeled data is called unsupervised feature learning, which plays an important role in pretraining a deep neural network. Here we provide a statistical mechanics analysis of the unsupervised learning in a restricted Boltzmann machine with binary synapses. A message passing equation to infer the hidden feature is derived, and furthermore, variants of this equation are analyzed. A statistical analysis by replica theory describes the thermodynamic properties of the model. Our analysis confirms an entropy crisis preceding the non-convergence of the message passing equation, suggesting a discontinuous phase transition as a key characteristic of the restricted Boltzmann machine. Continuous phase transition is also confirmed depending on the embedded feature strength in the data. The mean-field result under the replica symmetric assumption agrees with that obtained by running message passing algorithms on single instances of finite sizes. Interestingly, in an approximate Hopfield model, the entropy crisis is absent, and a continuous phase transition is observed instead. We also develop an iterative equation to infer the hyper-parameter (temperature) hidden in the data, which in physics corresponds to iteratively imposing Nishimori condition. Our study provides insights towards understanding the thermodynamic properties of the restricted Boltzmann machine learning, and moreover important theoretical basis to build simplified deep networks.

18 May 2015 in Physical Review E
Verified
Journal Article
Closed access
PDF file
HHHaiping Huang
Haiping Huang
TTTaro Toyoizumi
Taro Toyoizumi
Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean-field theory based on the Bethe approximation. Our theory provides an efficient message-passing-based method that evaluates not only the partition function (free energy) but also its gradients without requiring statistical sampling. The results are compared with those obtained by the computationally expensive sampling-based method.

14 December 2018 in Physical Review E
Verified
Journal Article
Closed access
PDF file
HHHaiping Huang
Haiping Huang
Deep neural networks are widely used in various domains. However, the nature of computations at each layer of the deep networks is far from being well understood. Increasing the interpretability of deep neural networks is thus important. Here, we construct a mean-field framework to understand how compact representations are developed across layers, not only in deterministic deep networks with random weights but also in generative deep networks where an unsupervised learning is carried out. Our theory shows that the deep computation implements a dimensionality reduction while maintaining a finite level of weak correlations between neurons for possible feature extraction. Mechanisms of dimensionality reduction and decorrelation are unified in the same framework. This work may pave the way for understanding how a sensory hierarchy works.