Xiaoxiao Jin

Mr. Xiaoxiao Jin

orcid

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 April 2024 in Symmetry
Verified
Journal Article
Open access
PDF file
XJXiaoxiao Jin
Xiaoxiao Jin
CMChenyang Mao
Chenyang Mao
DYDengfeng Yue
Dengfeng Yue
TLTuo Leng
Tuo Leng
The processing and comprehension of numerical information in natural language represent pivotal focal points of scholarly inquiry. Across diverse applications spanning text analysis to information retrieval, the adept management and understanding of the numerical content within natural language are indispensable in achieving task success. Specialized encoding and embedding techniques tailored to numerical data offer an avenue toward improved performance in tasks involving masked prediction and numerical reasoning, inherently characterized by numerical values. Consequently, treating numbers in text merely as words is inadequate; their numerical semantics must be underscored. Recent years have witnessed the emergence of a range of specific encoding methodologies designed explicitly for numerical content, demonstrating promising outcomes. We observe similarities between the Transformer architecture and CPU architecture, with symmetry playing a crucial role. In light of this observation and drawing inspiration from computer system theory, we introduce a floating-point representation and devise a corresponding embedding module. The numerical representations correspond one-to-one with their semantic vector values, rendering both symmetric regarding intermediate transformation methods. Our proposed methodology facilitates the more comprehensive encoding and embedding of numerical information within a predefined precision range, thereby ensuring a distinctive encoding representation for each numerical entity. Rigorous testing on multiple encoder-only models and datasets yielded results that stand out in terms of competitiveness. In comparison to the default embedding methods employed by models, our approach achieved an improvement of approximately 3.8% in Top-1 accuracy and a reduction in perplexity of approximately 0.43. These outcomes affirm the efficacy of our proposed method. Furthermore, the enrichment of numerical semantics through a more comprehensive embedding contributes to the augmentation of the model’s capacity for semantic understanding.

17 November 2023
Verified
Conference Paper
Closed access
ZZZhe Zhu
Zhe Zhu
XJXiaoxiao Jin
Xiaoxiao Jin
JZJia Zou
Jia Zou
TLTuo Leng
Tuo Leng
The past years have witnessed remarkable achievements in large-scale deep models. Nevertheless, their broad implementation on resource-constrained devices faces significant obstacles due to considerable computational complexity and extensive storage demands. To address these challenges, various approaches, such as model compression and neural architecture design, have been proposed. GhostNet, a lightweight network architecture, exploits the redundancy of feature maps by categorizing convolution filters into primary and cheap filters. Nonetheless, the limited number of primary filters, coupled with a high compression ratio, restrains the network’s expressiveness. Drawing inspiration from regularization methods targeting the angular diversity of neurons, we introduce a novel approach to augment GhostNet’s performance. Specifically, we propose the integration of maximizing minimal pairwise angles (MMA) regularization on primary filters. This augmentation aims to enhance the disparity between convolution kernels, thereby extracting more information at different levels. The MMA regularization strives to uniformly distribute the normalized weight vectors of neurons or filters on a hypersphere. Our experimental results, conducted on benchmark datasets, showcase the efficacy of the proposed method in enhancing GhostNet’s performance, particularly under higher compression ratios. This advancement opens new avenues for improving the deployment of deep models on resource-constrained devices.