Yann LeCun

Yann LeCun

Auto-generated profile, based on publicly available information

Yann LeCun

Facebook AI Research

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 November 2022
Verified
Book Chapter
Closed access
PDF file
CYChun-Hsiao Yeh
Chun-Hsiao Yeh
CHCheng-Yao Hong
Cheng-Yao Hong
YHYen-Chi Hsu
Yen-Chi Hsu
TLTyng-Luh Liu
Tyng-Luh Liu
YCYubei Chen
Yubei Chen
YLYann LeCun
Yann LeCun

1 May 2016 in Neural Computation
Verified
Journal Article
Closed access
MTMark Tygert
Mark Tygert
JBJoan Bruna
Joan Bruna
SCSoumith Chintala
Soumith Chintala
YLYann LeCun
Yann LeCun
SPSerkan Piantino
Serkan Piantino
ASArthur Szlam
Arthur Szlam
A complex-valued convolutional network (convnet) implements the repeated application of the following composition of three operations, recursively applying the composition to an input vector of nonnegative real numbers: (1) convolution with complex-valued vectors, followed by (2) taking the absolute value of every entry of the resulting vectors, followed by (3) local averaging. For processing real-valued random vectors, complex-valued convnets can be viewed as data-driven multiscale windowed power spectra, data-driven multiscale windowed absolute spectra, data-driven multiwavelet absolute values, or (in their most general configuration) data-driven nonlinear multiwavelet packets. Indeed, complex-valued convnets can calculate multiscale windowed spectra when the convnet filters are windowed complex-valued exponentials. Standard real-valued convnets, using rectified linear units (ReLUs), sigmoidal (e.g., logistic or tanh) nonlinearities, or max pooling, for example, do not obviously exhibit the same exact correspondence with data-driven wavelets (whereas for complex-valued convnets, the correspondence is much more than just a vague analogy). Courtesy of the exact correspondence, the remarkably rich and rigorous body of mathematical analysis for wavelets applies directly to (complex-valued) convnets.

2 November 2018 in Research-Technology Management
Verified
Journal Article
Closed access
YLYann LeCun
Yann LeCun
(2018). The Power and Limits of Deep Learning. Research-Technology Management: Vol. 61, No. 6, pp. 22-27.

14 June 2021 in arXiv
Verified
Preprint
Open access
PDF file
JZJure Zbontar
Jure Zbontar
LJLi Jing
Li Jing
IMIshan Misra
Ishan Misra
YLYann LeCun
Yann LeCun
SDStéphane Deny
Stéphane Deny
Self-supervised learning (SSL) is rapidly closing the gap with supervised methods on large computer vision benchmarks. A successful approach to SSL is to learn embeddings which are invariant to distortions of the input sample. However, a recurring issue with this approach is the existence of trivial constant solutions. Most current methods avoid such solutions by careful implementation details. We propose an objective function that naturally avoids collapse by measuring the cross-correlation matrix between the outputs of two identical networks fed with distorted versions of a sample, and making it as close to the identity matrix as possible. This causes the embedding vectors of distorted versions of a sample to be similar, while minimizing the redundancy between the components of these vectors. The method is called Barlow Twins, owing to neuroscientist H. Barlow's redundancy-reduction principle applied to a pair of identical networks. Barlow Twins does not require large batches nor asymmetry between the network twins such as a predictor network, gradient stopping, or a moving average on the weight updates. Intriguingly it benefits from very high-dimensional output vectors. Barlow Twins outperforms previous methods on ImageNet for semi-supervised classification in the low-data regime, and is on par with current state of the art for ImageNet classification with a linear classifier head, and for transfer tasks of classification and object detection.

27 January 2017 in arXiv
Verified
Preprint
Open access
PDF file
ACAlexis Conneau
Alexis Conneau
HSHolger Schwenk
Holger Schwenk
LBLoïc Barrault
Loïc Barrault
YLYann Lecun
Yann Lecun
The dominant approach for many NLP tasks are recurrent neural networks, in particular LSTMs, and convolutional neural networks. However, these architectures are rather shallow in comparison to the deep convolutional networks which have pushed the state-of-the-art in computer vision. We present a new architecture (VDCNN) for text processing which operates directly at the character level and uses only small convolutions and pooling operations. We are able to show that the performance of this model increases with depth: using up to 29 convolutional layers, we report improvements over the state-of-the-art on several public text classification tasks. To the best of our knowledge, this is the first time that very deep convolutional nets have been applied to text processing.

13 October 2022 in arXiv
Verified
Preprint
Open access
PDF file
BKBobak Kiani
Bobak Kiani
RBRandall Balestriero
Randall Balestriero
YLYann LeCun
Yann LeCun
SLSeth Lloyd
Seth Lloyd
In learning with recurrent or very deep feed-forward networks, employing unitary matrices in each layer can be very effective at maintaining long-range stability. However, restricting network parameters to be unitary typically comes at the cost of expensive parameterizations or increased training runtime. We propose instead an efficient method based on rank-$k$ updates -- or their rank-$k$ approximation -- that maintains performance at a nearly optimal training runtime. We introduce two variants of this method, named Direct (projUNN-D) and Tangent (projUNN-T) projected Unitary Neural Networks, that can parameterize full $N$-dimensional unitary or orthogonal matrices with a training runtime scaling as $O(kN^2)$. Our method either projects low-rank gradients onto the closest unitary matrix (projUNN-T) or transports unitary matrices in the direction of the low-rank gradient (projUNN-D). Even in the fastest setting ($k=1$), projUNN is able to train a model's unitary parameters to reach comparable performances against baseline implementations. In recurrent neural network settings, projUNN closely matches or exceeds benchmarked results from prior unitary neural networks. Finally, we preliminarily explore projUNN in training orthogonal convolutional neural networks, which are currently unable to outperform state of the art models but can potentially enhance stability and robustness at large depth.