Susan Zhang

Susan Zhang

Auto-generated profile, based on publicly available information

Susan Zhang

Meta AI Meta Platforms, Inc

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 April 2023 in arXiv
Verified
Preprint
Open access
PDF file
EDEmily Dinan
Emily Dinan
SYSho Yaida
Sho Yaida
SZSusan Zhang
Susan Zhang
We perform an effective-theory analysis of forward-backward signal propagation in wide and deep Transformers, i.e., residual neural networks with multi-head self-attention blocks and multilayer perceptron blocks. This analysis suggests particular width scalings of initialization and training hyperparameters for these models. We then take up such suggestions, training Vision and Language Transformers in practical setups.