Shuming Ma

Shuming Ma

Auto-generated profile, based on publicly available information

Shuming Ma

Microsoft Research Asia

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 April 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
HWHongyu Wang
Hongyu Wang
SMShuming Ma
Shuming Ma
LDLi Dong
Li Dong
SHShaohan Huang
Shaohan Huang
DZDongdong Zhang
Dongdong Zhang
FWFuru Wei
Furu Wei
In this paper, we propose a simple yet effective method to stabilize extremely deep Transformers. Specifically, we introduce a new normalization function ( DeepNorm ) to modify the residual connection in Transformer, accompanying with theoretically derived initialization. In-depth theoretical analysis shows that model updates can be bounded in a stable way. The proposed method combines the best of two worlds, i.e., good performance of Post-LN and stable training of Pre-LN, making DeepNorm a preferred alternative. We successfully scale Transformers up to 1,000 layers (i.e., 2,500 attention and feed-forward network sublayers) without difficulty, which is one order of magnitude deeper than previous deep Transformers. Extensive experiments demonstrate that DeepNet has superior performance across various benchmarks, including machine translation, language modeling (i.e., BERT, GPT) and vision pre-training (i.e., BEiT). Remarkably, on a multilingual benchmark with 7,482 translation directions, our 200-layer model with 3.2B parameters significantly outperforms the 48-layer state-of-the-art model with 12B parameters by 5 BLEU points, which indicates a promising scaling direction.

15 April 2023
Verified
Book Chapter
Closed access
PDF file
JYJian Yang
Jian Yang
YYYuwei Yin
Yuwei Yin
SMShuming Ma
Shuming Ma
LYLiqun Yang
Liqun Yang
HGHugo Guo
Hugo Guo
FWFuru Wei
Furu Wei

1 November 2026 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
HWHongyu Wang
Hongyu Wang
SMShuming Ma
Shuming Ma
FWFuru Wei
Furu Wei
Efficient deployment of 1-bit Large Language Models (LLMs) is hindered by activation outliers, which pose significant challenges to low-bit quantization. In this work, we investigate sparse and low-bit activation strategies tailored for 1-bit LLMs. We first introduce BitNet v2 a4.8, a hybrid quantization-sparsification architecture designed to mitigate quantization errors arising from outlier activation channels. BitNet v2 a4.8 utilizes 4-bit quantization for inputs to attention and feed-forward layers, while sparsifying intermediate activations and subsequently quantizing them to 8 bits. To further leverage the 4-bit computation capabilities of next-generation GPUs, we present BitNet v2, a novel framework that enables fully 4-bit activations for 1-bit LLMs. To address the outlier activation channels, we propose $/mathcal{H}$, a lightweight module that applies an online Hadamard transformation prior to quantization, effectively regularizing activation distributions into more Gaussian-like forms conducive to low-bit representation. Extensive experiments demonstrate that BitNet v2 a4.8 achieves performance comparable to BitNet b1.58 with similar training costs. Furthermore, BitNet v2, when trained from scratch with 8-bit activations, matches the performance of BitNet b1.58. Remarkably, training BitNet v2 directly with native 4-bit activations leads to only marginal performance degradation, while significantly reducing computational overhead during batched inference.

1 January 2021
Verified
Conference Paper
Open access
PDF file
JYJian Yang
Jian Yang
SMShuming Ma
Shuming Ma
DZDongdong Zhang
Dongdong Zhang
JWJuncheng Wan
Juncheng Wan
ZLZhoujun Li
Zhoujun Li
MZMing Zhou
Ming Zhou
Jian Yang, Shuming Ma, Dongdong Zhang, Juncheng Wan, Zhoujun Li, Ming Zhou. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

7 November 2022 in arXiv
Verified
Preprint
Open access
PDF file
JYJian Yang
Jian Yang
YYYuwei Yin
Yuwei Yin
LYLiqun Yang
Liqun Yang
SMShuming Ma
Shuming Ma
HHHaoyang Huang
Haoyang Huang
ZLZhoujun Li
Zhoujun Li
Transformer structure, stacked by a sequence of encoder and decoder network layers, achieves significant development in neural machine translation. However, vanilla Transformer mainly exploits the top-layer representation, assuming the lower layers provide trivial or redundant information and thus ignoring the bottom-layer feature that is potentially valuable. In this work, we propose the Group-Transformer model (GTrans) that flexibly divides multi-layer representations of both encoder and decoder into different groups and then fuses these group features to generate target words. To corroborate the effectiveness of the proposed method, extensive experiments and analytic experiments are conducted on three bilingual translation benchmarks and two multilingual translation tasks, including the IWLST-14, IWLST-17, LDC, WMT-14 and OPUS-100 benchmark. Experimental and analytical results demonstrate that our model outperforms its Transformer counterparts by a consistent gain. Furthermore, it can be successfully scaled up to 60 encoder layers and 36 decoder layers.

5 November 2021 in arXiv
Verified
Preprint
Open access
PDF file
GCGuanhua Chen
Guanhua Chen
SMShuming Ma
Shuming Ma
YCYun Chen
Yun Chen
LDLi Dong
Li Dong
DZDongdong Zhang
Dongdong Zhang
FWFuru Wei
Furu Wei
Previous work mainly focuses on improving cross-lingual transfer for NLU tasks with a multilingual pretrained encoder (MPE), or improving the performance on supervised machine translation with BERT. However, it is under-explored that whether the MPE can help to facilitate the cross-lingual transferability of NMT model. In this paper, we focus on a zero-shot cross-lingual transfer task in NMT. In this task, the NMT model is trained with parallel dataset of only one language pair and an off-the-shelf MPE, then it is directly tested on zero-shot language pairs. We propose SixT, a simple yet effective model for this task. SixT leverages the MPE with a two-stage training schedule and gets further improvement with a position disentangled encoder and a capacity-enhanced decoder. Using this method, SixT significantly outperforms mBART, a pretrained multilingual encoder-decoder model explicitly designed for NMT, with an average improvement of 7.1 BLEU on zero-shot any-to-English test sets across 14 source languages. Furthermore, with much less training computation cost and training data, our model achieves better performance on 15 any-to-English test sets than CRISS and m2m-100, two strong multilingual NMT baselines.