With the continuous growth in the number of parameters of the Transformer-based pretrained language models (PLMs), particularly the emergence of large language models (LLMs) with billions of parameters, many natural language processing (NLP) tasks have demonstrated remarkable success. However, the enormous size and computational demands of these models pose significant challenges for adapting them to specific downstream tasks, especially in environments with limited computational resources. Parameter-Efficient Fine-Tuning (PEFT) offers an effective solution by reducing the number of fine-tuning parameters and memory usage while achieving comparable performance to full fine-tuning. The demands for fine-tuning PLMs, especially LLMs, have led to a surge in the development of PEFT methods, as depicted in Fig. 1. In this paper, we present a comprehensive and systematic review of PEFT methods for PLMs. We summarize these PEFT methods, discuss their applications, and outline future directions. Furthermore, extensive experiments are conducted using several representative PEFT methods to better understand their effectiveness in parameter efficiency and memory efficiency. By offering insights into the latest advancements and practical applications, this survey serves as an invaluable resource for researchers and practitioners seeking to navigate the challenges and opportunities presented by PEFT in the context of PLMs.

While the wisdom of training an image dehazing model on synthetic hazy data can alleviate the difficulty of collecting real-world hazy/clean image pairs, it brings the well-known domain shift problem. From a different yet new perspective, this paper explores contrastive learning with an adversarial training effort to leverage unpaired real-world hazy and clean images, thus alleviating the domain shift problem and enhancing the network’s generalization ability in real-world scenarios. We propose an effective unsupervised contrastive learning paradigm for image dehazing, dubbed UCL-Dehaze. Unpaired real-world clean and hazy images are easily captured, and will serve as the important positive and negative samples respectively when training our UCL-Dehaze network. To train the network more effectively, we formulate a new self-contrastive perceptual loss function, which encourages the restored images to approach the positive samples and keep away from the negative samples in the embedding space. Besides the overall network architecture of UCL-Dehaze, adversarial training is utilized to align the distributions between the positive samples and the dehazed images. Compared with recent image dehazing works, UCL-Dehaze does not require paired data during training and utilizes unpaired positive/negative data to better enhance the dehazing performance. We conduct comprehensive experiments to evaluate our UCL-Dehaze and demonstrate its superiority over the state-of-the-arts, even only 1,800 unpaired real-world images are used to train our network. Source code is publicly available at https://github.com/yz-wang/UCL-Dehaze.

Research on Educational Metaverse (Edu-Metaverse) has developed into an active research field. Based on 310 academic papers published from 2004 to 2022, this study identifies contributors, scientific cooperations, and research themes using bibliometrics, social network analysis, topic modeling, and keyword analysis. Results suggest that Edu-Metaverse has gained increasing attention in academia since 2019. Countries/affiliations located in the same regions are close partners in scientific cooperation. By jointly interpreting topic modeling and keyword analysis results, this study reveals eight main themes in the field of Edu-Metaverse: 1) metaverse-based physical education, 2) metaverses-supported simulations for collaborative problem-based learning (PBL) in health/medical education, 3) three-dimensional virtual learning environment-supported art appreciation and creation in art education, 4) metaverse-enabled laboratories for STEM education, 5) language and 21st-century skill development through metaverse-supported immersive language learning, 6) metaverse for developing autism children' social communication abilities, 7) virtual world metaverse-supported gameful experience-based education, and 8) quantitative research on Edu-Metaverse focusing on learners' experience. We also identified challenges and directions needing further attention: 1) data security and privacy protection, 2) balance between the real world and virtual world identities, 3) preparing instructors for Edu-Metaverse, and 4) assessment of higher-order thinking competencies in Edu-Metaverse-based PBL. This work helps facilitate researchers' and practitioners' understanding of Edu-Metaverse research and raises their awareness of research frontiers and future directions.

Natural language processing (NLP) has become an increasingly significant role in advancing medicine. Rich research achievements of NLP methods and applications for medical information processing are available. It is of great significance to conduct a deep analysis to understand the recent development of NLP-empowered medical research field. However, limited study examining the research status of this field could be found. Therefore, this study aims to quantitatively assess the academic output of NLP in medical research field. We conducted a bibliometric analysis on NLP-empowered medical research publications retrieved from PubMed in the period 2007–2016. The analysis focused on three aspects. Firstly, the literature distribution characteristics were obtained with a statistics analysis method. Secondly, a network analysis method was used to reveal scientific collaboration relations. Finally, thematic discovery and evolution was reflected using an affinity propagation clustering method. There were 1405 NLP-empowered medical research publications published during the 10 years with an average annual growth rate of 18.39%. 10 most productive publication sources together contributed more than 50% of the total publications. The USA had the highest number of publications. A moderately significant correlation between country’s publications and GDP per capita was revealed. Denny, Joshua C was the most productive author. Mayo Clinic was the most productive affiliation. The annual co-affiliation and co-country rates reached 64.04% and 15.79% in 2016, respectively. 10 main great thematic areas were identified including Computational biology, Terminology mining, Information extraction, Text classification, Social medium as data source, Information retrieval, etc. A bibliometric analysis of NLP-empowered medical research publications for uncovering the recent research status is presented. The results can assist relevant researchers, especially newcomers in understanding the research development systematically, seeking scientific cooperation partners, optimizing research topic choices and monitoring new scientific or technological activities.

The pretrained large language models have been widely tested for their performance on some challenging tasks including arithmetic, commonsense, and symbolic reasoning. Recently how to combine LLMs with prompting techniques has attracted lots of researchers to propose their models to automatically solve math word problems. However, most research works focus on solving math problems at the elementary school level and few works aim to solve problems in science disciplines, e.g., Physics. In this exploratory study, we discussed the potential pedagogical benefits of using ChatGPT in physics and demonstrated how to prompt ChatGPT in solving physics problems. The results suggest that ChatGPT is able to solve some physics calculation problems, explain solutions, and generate new exercises at a human level.

Peer, teacher, and self-feedback have been widely applied in English writing courses in higher education. However, few studies have used technology to activate the potential of feedback in project-based collaborative learning or discussed how technology-enhanced peer, teacher and self-feedback may assist students’ writing, promote their critical thinking tendency, or enhance their engagement in learning, so we investigated them in this research. A total of 90 students, 30 in each group, participated in it. They reported their progress at four stages every other week, received peer, teacher, and self-feedback respectively for 10 weeks, and submitted their finalized review articles in week 14. Before the treatment, we evaluated the students’ writing proficiency and critical thinking tendency through a pre-test and a pre-questionnaire survey. After the treatment, we evaluated their collaborative writing products and conducted a post-questionnaire survey to measure their critical thinking tendency and behavioral, cognitive, and emotional engagement in learning. The results indicated that technology-enhanced peer and teacher feedback were significantly more effective than self-feedback in assisting collaborative writing; peer and self-feedback were significantly more effective than teacher feedback in promoting critical thinking tendency, enhancing behavioral and emotional engagement in learning; and teacher feedback was significantly more effective than self-feedback in enhancing cognitive engagement in learning. We also conducted semi-structured interviews to investigate their perception of the three feedback types and the technology-enhanced feedback-assisted collaborative writing experience. Most students enjoyed the writing experience and regarded the use of digital tools effective for its implementation. Based on these results, we suggest that teachers implement more technology-enhanced peer and self-feedback assisted collaborative writing.