Zohar Ringel

Zohar Ringel

Auto-generated profile, based on publicly available information

Zohar Ringel

The Racah Institute of Physics, The Hebrew University of Jerusalem, Jerusalem, Israel

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 September 2017 in Science Advances
Verified
Journal Article
Open access
PDF file
ZRZohar Ringel
Zohar Ringel
DKDmitry L. Kovrizhin
Dmitry L. Kovrizhin
Energy transport in metals helps uncover a link between Einstein’s gravity and quantum computational complexity.

9 April 2021 in Physical Review Research
Verified
Journal Article
Open access
PDF file
OCOmry Cohen
Omry Cohen
OMOr Malka
Or Malka
ZRZohar Ringel
Zohar Ringel
In the past decade, deep neural networks (DNNs) came to the fore as the leading machine-learning algorithms for a variety of tasks. Their rise was founded on market needs and engineering craftsmanship, the latter based more on trial and error than on theory. While still far behind the application forefront, the theoretical study of DNNs has recently made important advancements in analyzing the highly overparametrized regime where some exact results have been obtained. Leveraging these ideas and adopting a more physicslike approach, here we construct a versatile field theory formalism for supervised deep learning, involving renormalization group, Feynman diagrams, and replicas. In particular, we show that our approach leads to highly accurate predictions of learning curves of truly deep DNNs trained on polynomial regression problems. It also explains in a concrete manner why DNNs generalize well despite being highly overparametrized, this due to an entropic bias to simple functions which, for the case of fully connected DNNs with data sampled on the hypersphere, are low-order polynomials in the input vector. Being a complex interacting system of artificial neurons, we believe that such tools and methodologies borrowed from condensed matter physics would prove essential for obtaining an accurate quantitative understanding of deep learning.

17 June 2021 in Physical Review Letters
Verified
Journal Article
Closed access
PDF file
AGAmit Gordon
Amit Gordon
ABAditya Banerjee
Aditya Banerjee
MKMaciej Koch-Janusz
Maciej Koch-Janusz
ZRZohar Ringel
Zohar Ringel
The analysis of complex physical systems hinges on the ability to extract the relevant degrees of freedom from among the many others. Though much hope is placed in machine learning, it also brings challenges, chief of which is interpretability. It is often unclear what relation, if any, the architecture- and training-dependent learned “relevant” features bear to standard objects of physical theory. Here we report on theoretical results which may help to systematically address this issue: we establish equivalence between the field-theoretic relevance of the renormalization group, and an information-theoretic notion of relevance we define using the information bottleneck (IB) formalism of compression theory. We show analytically that for statistical physical systems described by a field theory the relevant degrees of freedom found using IB compression indeed correspond to operators with the lowest scaling dimensions. We confirm our field theoretic predictions numerically. We study dependence of the IB solutions on the physical symmetries of the data. Our findings provide a dictionary connecting two distinct theoretical toolboxes, and an example of constructively incorporating physical interpretability in applications of deep learning in physics.

21 May 2025 in Machine Learning: Science and Technology
Verified
Journal Article
Open access
JHJessica N Howard
Jessica N Howard
RJRo Jefferson
Ro Jefferson
Anindita MaitiAnindita Maiti
Anindita Maiti
ZRZohar Ringel
Zohar Ringel
Separating relevant and irrelevant information is key to any modeling process or scientific inquiry. Theoretical physics offers a powerful tool for achieving this in the form of the renormalization group (RG). Here we demonstrate a practical approach to performing Wilsonian RG in the context of Gaussian Process (GP) Regression. We systematically integrate out the unlearnable modes of the GP kernel, thereby obtaining an RG flow of the GP in which the data sets the IR scale. In simple cases, this results in a universal flow of the ridge parameter, which becomes input-dependent in the richer scenario in which non-Gaussianities are included. In addition to being analytically tractable, this approach goes beyond structural analogies between RG and neural networks by providing a natural connection between RG flow and learnable vs. unlearnable modes. Studying such flows may improve our understanding of feature learning in deep neural networks, and enable us to identify potential universality classes in these models.

26 November 2020 in arXiv
Verified
Preprint
Open access
PDF file
OCOmry Cohen
Omry Cohen
OMOr Malka
Or Malka
ZRZohar Ringel
Zohar Ringel
In the past decade, deep neural networks (DNNs) came to the fore as the leading machine learning algorithms for a variety of tasks. Their raise was founded on market needs and engineering craftsmanship, the latter based more on trial and error than on theory. While still far behind the application forefront, the theoretical study of DNNs has recently made important advancements in analyzing the highly over-parameterized regime where some exact results have been obtained. Leveraging these ideas and adopting a more physics-like approach, here we construct a versatile field-theory formalism for supervised deep learning, involving renormalization group, Feynman diagrams and replicas. In particular we show that our approach leads to highly accurate predictions of learning curves of truly deep DNNs trained on polynomial regression tasks and that these predictions can be used for efficient hyper-parameter optimization. In addition, they explain how DNNs generalize well despite being highly over-parameterized, this due to an entropic bias to simple functions which, for the case of fully-connected DNNs with data sampled on the hypersphere, are low order polynomials in the input vector. Being a complex interacting system of artificial neurons, we believe that such tools and methodologies borrowed from condensed matter physics would prove essential for obtaining an accurate quantitative understanding of deep learning.

6 February 2018 in arXiv
Verified
Preprint
Open access
PDF file
ZRZohar Ringel
Zohar Ringel
RBRodrigo de Bem
Rodrigo de Bem
In this paper we approach two relevant deep learning topics: i) tackling of graph structured input data and ii) a better understanding and analysis of deep networks and related learning algorithms. With this in mind we focus on the topological classification of reachability in a particular subset of planar graphs (Mazes). Doing so, we are able to model the topology of data while staying in Euclidean space, thus allowing its processing with standard CNN architectures. We suggest a suitable architecture for this problem and show that it can express a perfect solution to the classification task. The shape of the cost function around this solution is also derived and, remarkably, does not depend on the size of the maze in the large maze limit. Responsible for this behavior are rare events in the dataset which strongly regulate the shape of the cost function near this global minimum. We further identify an obstacle to learning in the form of poorly performing local minima in which the network chooses to ignore some of the inputs. We further support our claims with training experiments and numerical analysis of the cost function on networks with up to $128$ layers.