Ming Zhou

Ming Zhou

Auto-generated profile, based on publicly available information

Ming Zhou

Langboat Technology

Research Focus & Metrics

Academic Career Length

25Years (2002-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 October 2023 in ACM Computing Surveys
Verified
Journal Article
Open access
HZHanqing Zhang
Hanqing Zhang
HSHaolin Song
Haolin Song
SLShaoyu Li
Shaoyu Li
MZMing Zhou
Ming Zhou
DSDawei Song
Dawei Song
Controllable Text Generation (CTG) is an emerging area in the field of natural language generation (NLG). It is regarded as crucial for the development of advanced text generation technologies that better meet the specific constraints in practical applications. In recent years, methods using large-scale pre-trained language models (PLMs), in particular the widely used Transformer-based PLMs, have become a new paradigm of NLG, allowing generation of more diverse and fluent text. However, due to the limited level of interpretability of deep neural networks, the controllability of these methods needs to be guaranteed. To this end, controllable text generation using Transformer-based PLMs has become a rapidly growing yet challenging new research hotspot. A diverse range of approaches have emerged in the past 3 to 4 years, targeting different CTG tasks that require different types of controlled constraints. In this article, we present a systematic critical review on the common tasks, main approaches, and evaluation methods in this area. Finally, we discuss the challenges that the field is facing, and put forward various promising future directions. To the best of our knowledge, this is the first survey article to summarize the state-of-the-art CTG techniques from the perspective of Transformer-based PLMs. We hope it can help researchers and practitioners in the related fields to quickly track the academic and technological frontier, providing them with a landscape of the area and a roadmap for future research.

6 July 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
HLHanmeng Liu
Hanmeng Liu
JLJian Liu
Jian Liu
LCLeyang Cui
Leyang Cui
ZTZhiyang Teng
Zhiyang Teng
NDNan Duan
Nan Duan
YZYue Zhang
Yue Zhang
NLP research on logical reasoning regains momentum with the recent releases of a handful of datasets, notably LogiQA and Reclor. Logical reasoning is exploited in many probing tasks over large Pre-trained Language Models (PLMs) and downstream tasks like question-answering and dialogue systems. In this paper, we release LogiQA 2.0. The dataset is an amendment and re-annotation of LogiQA in 2020, a large-scale logical reasoning reading comprehension dataset adapted from the Chinese Civil Service Examination. We increase the data size, refine the texts with manual translation by professionals, and improve the quality by removing items with distinctive cultural features like Chinese idioms. Furthermore, we conduct a fine-grained annotation on the dataset and turn it into a two-way natural language inference (NLI) task, resulting in 35k premise-hypothesis pairs with gold labels, making it the first large-scale NLI dataset for complex logical reasoning. Compared to Question Answering, Natural Language Inference excels in generalizability and helps downstream tasks better. We establish a baseline for logical reasoning in NLI and incite further research.

1 April 2022 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
SWSiyuan Wang
Siyuan Wang
ZLZhongkun Liu
Zhongkun Liu
WZWanjun Zhong
Wanjun Zhong
MZMing Zhou
Ming Zhou
ZWZhongyu Wei
Zhongyu Wei
NDNan Duan
Nan Duan
Complex reasoning aims to draw a correct inference based on complex rules. As a hallmark of human intelligence, it involves a degree of explicit reading comprehension, interpretation of logical knowledge and complex rule application. In this paper, we take a step forward in complex reasoning by systematically studying the three challenging and domain-general tasks of the Law School Admission Test (LSAT), including analytical reasoning, logical reasoning and reading comprehension. We propose a hybrid reasoning system to integrate these three tasks and achieve impressive overall performance on the LSAT tests. The experimental results demonstrate that our system endows itself a certain complex reasoning ability, especially the fundamental reading comprehension and challenging logical reasoning capacities. Further analysis also shows the effectiveness of combining the pre-trained models with the task-specific reasoning module, and integrating symbolic knowledge into discrete interpretable reasoning steps in complex reasoning. We further shed a light on the potential future directions, like unsupervised symbolic knowledge extraction, model interpretability, few-shot learning and comprehensive benchmark for complex reasoning.

6 October 2021
Verified
Book Chapter
Closed access
PDF file
QXQiaolin Xia
Qiaolin Xia
HHHaoyang Huang
Haoyang Huang
NDNan Duan
Nan Duan
DZDongdong Zhang
Dongdong Zhang
LJLei Ji
Lei Ji
MZMing Zhou
Ming Zhou

1 January 2019
Verified
Conference Paper
Open access
PDF file
JXJingjing Xu
Jingjing Xu
YWYuechen Wang
Yuechen Wang
DTDuyu Tang
Duyu Tang
NDNan Duan
Nan Duan
PYPengcheng Yang
Pengcheng Yang
XSXu Sun
Xu Sun
Jingjing Xu, Yuechen Wang, Duyu Tang, Nan Duan, Pengcheng Yang, Qi Zeng, Ming Zhou, Xu SUN. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

6 April 2020 in arXiv
Verified
Preprint
Open access
PDF file
WWWenhui Wang
Wenhui Wang
FWFuru Wei
Furu Wei
LDLi Dong
Li Dong
HBHangbo Bao
Hangbo Bao
NYNan Yang
Nan Yang
MZMing Zhou
Ming Zhou
Pre-trained language models (e.g., BERT (Devlin et al., 2018) and its variants) have achieved remarkable success in varieties of NLP tasks. However, these models usually consist of hundreds of millions of parameters which brings challenges for fine-tuning and online serving in real-life applications due to latency and capacity constraints. In this work, we present a simple and effective approach to compress large Transformer (Vaswani et al., 2017) based pre-trained models, termed as deep self-attention distillation. The small model (student) is trained by deeply mimicking the self-attention module, which plays a vital role in Transformer networks, of the large model (teacher). Specifically, we propose distilling the self-attention module of the last Transformer layer of the teacher, which is effective and flexible for the student. Furthermore, we introduce the scaled dot-product between values in the self-attention module as the new deep self-attention knowledge, in addition to the attention distributions (i.e., the scaled dot-product of queries and keys) that have been used in existing works. Moreover, we show that introducing a teacher assistant (Mirzadeh et al., 2019) also helps the distillation of large pre-trained Transformer models. Experimental results demonstrate that our monolingual model outperforms state-of-the-art baselines in different parameter size of student models. In particular, it retains more than 99% accuracy on SQuAD 2.0 and several GLUE benchmark tasks using 50% of the Transformer parameters and computations of the teacher model. We also obtain competitive results in applying deep self-attention distillation to multilingual pre-trained models.