Siddharth Goyal

Siddharth Goyal

Auto-generated profile, based on publicly available information

Siddharth Goyal

Facebook AI Research, USA

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 April 2019 in bioRxiv
Verified
Preprint
Open access
PDF file
ARAlexander Rives
Alexander Rives
JMJoshua Meier
Joshua Meier
TSTom Sercu
Tom Sercu
SGSiddharth Goyal
Siddharth Goyal
ZLZeming Lin
Zeming Lin
RFRob Fergus
Rob Fergus
In the field of artificial intelligence, a combination of scale in data and model capacity enabled by un-supervised learning has led to major advances in representation learning and statistical generation. In the life sciences, the anticipated growth of sequencing promises unprecedented data on natural sequence diversity. Protein language modeling at the scale of evolution is a logical step toward predictive and generative artificial intelligence for biology. To this end we use unsupervised learning to train a deep contextual language model on 86 billion amino acids across 250 million protein sequences spanning evolutionary diversity. The resulting model contains information about biological properties in its representations. The representations are learned from sequence data alone. The learned representation space has a multi-scale organization reflecting structure from the level of biochemical properties of amino acids to remote homology of proteins. Information about secondary and tertiary structure is encoded in the representations and can be identified by linear projections. Representation learning produces features that generalize across a range of applications, enabling state-of-the-art supervised prediction of mutational effect and secondary structure, and improving state-of-the-art features for long-range contact prediction.