A crucial component of author profiling is language variety prediction, which looks for stylistic and regional differences in language usage in written texts. Applications for this activity are substantial in fields including localization, forensics, and customized marketing. Conventional methods depend on manually created features or embeddings from separate models, which might not fully represent the nuanced linguistic differences. In order to improve language variety prediction performance, we presented a hybrid model in this work that integrates statistical features and embeddings from two sophisticated transformer models, such as T5 (Text-to-Text Transfer Transformer) and BERT (Bidirectional Encoder Representations from Transformers). Different statistical features identified in this work are Lexical features, Syntactic features, Orthographic and Phonological features, Stylistic features, Semantic features, Pragmatic and discourse-level features and Temporal features. The embeddings from both models are projected into a shared dimensions space, averaged, appended, and utilized as input to a downstream classifier in order to accomplish effective integration. This method makes use of T5's generative capabilities and BERT's contextual richness to better capture syntactic and semantic patterns in text. The English language diversity dataset from the PAN 2017 competition is used to test the suggested framework. According to experimental evaluations, the hybrid embedding technique achieves better accuracy and Fl-score for language variety prediction, outperforming individual transformer models by a wide margin. When compared to other models for language variety prediction, the XGBoost classifier with Statistical Features, BERT, and T5 embeddings performed better in terms of Fl-Score (93.4%) and Accuracy (93.5%). These results demonstrate the importance of including complimentary transformer embeddings to improve author profiling tasks' state-of-the-art performance.
Author Profiling (AP) is a task of determining the demographic features like Age, Gender, Nativity language, location, Personality traits etc. about the author of a document. AP is used in different applications such as marketing linguistic profile, security and forensic science. The researchers proposed different types of solutions to author profiling based on stylistic features, content based features and deep learning techniques. The content based features proved their significance to improve the performance of author profiles prediction. Several approaches faced a problem of high dimensionality of features when experimented with content based features. The researchers used feature selection algorithms to identify the important features for experimentation. In this work, a Term Weight Measure (TWM) based approach is proposed for author profiling problem. In this approach, the important features are identified by using Feature Selection (FS) algorithm. After features are identified, the next important task is representation of document with identified features. The documents are represented as vectors and computation of each feature value in the vector representation is another important research task. TWMs are used to determine the importance of a feature in the vector representation. In the proposed approach, we proposed a new TWM based on the way the terms are distributed in corpus of documents. The proposed term weight measure performance is compared with different existing TWMs. The PAN competition 2014 reviews dataset is used for age and gender prediction of the author. Two Machine Learning (ML) algorithms such as Random Forest (RF) and Support Vector Machine (SVM) are used to evaluate the proposed term weight measure based approach. The experimental results attained in this work for age and gender prediction are good when compared with several popular solutions to author profiling.
13 July 2023 in International Journal on Recent and Innovation Trends in Computing and Communication
Author Profiling (AP) is a method of identifying the demographic profiles such as age, gender, location, native language and personality traits of an author by processing their written texts. The AP techniques are used in multiple applications such as literary research, marketing, forensics and security. The researchers identified various differences in the authors writing styles by analysing various datasets. The differences in writing styles are represented as stylistic features. The researchers extracted several style based features like structural, content, word, character, syntactic, readability and semantic features to recognize the profiles of the authors. Traditionally, the researchers extracted various feature combinations for differentiating the profiles of authors. Several existing works are used Machine Learning (ML) methods for predicting the author characteristics of a new author. The existing works achieved good accuracies for predicting the author characteristics by considering the both stylistic features and ML algorithms combination. Recently, in advent of Deep Learning (DL) techniques the researchers are proposed approaches to author profiling by using these techniques. Few researchers identified that the deep learning techniques performance is good for author profiles prediction than the results of style based features. In this work, a word embeddings based approach is proposed for gender and age prediction. In this approach, the experiment conducted with different word embedding models such as Word2Vec, GloVe, FastText and BERT for generating word vectors for words. The documents are converted as vectors by using the document representation technique which uses the word embeddings of words. The document vectors are transferred to three different ML algorithms such as Extreme Gradient Boosting (XGBoost), Random Forest (RF) and Logistic Regression (LR) for generating the trained model. This model is used for predicating the accuracy of age and gender prediction. The XGBoost classifier with word embeddings of BERT achieved good accuracies for age and gender prediction than other word embeddings and ML algorithms. The experiment implemented on PAN 2014 competition Reviews dataset for age and gender prediction. The proposed approach attained best accuracies for predicting age and gender than the performances of various existing approaches proposed for AP.