With the rapid proliferation of social networking sites (SNS), automatic topic extraction from various text messages posted on SNS are becoming an important source of information for understanding current social trends or needs. Latent Dirichlet Allocation (LDA), a probabilistic generative model, is one of the popular topic models in the area of Natural Language Processing (NLP) and has been widely used in information retrieval, topic extraction, and document analysis. Unlike long texts from formal documents, messages on SNS are generally short. Traditional topic models such as LDA or pLSA (probabilistic latent semantic analysis) suffer performance degradation for short-text analysis due to a lack of word co-occurrence information in each short text. To cope with this problem, various techniques are evolving for interpretable topic modeling for short texts, pretrained word embedding with an external corpus combined with topic models is one of them. Due to recent developments of deep neural networks (DNN) and deep generative models, neural-topic models (NTM) are emerging to achieve flexibility and high performance in topic modeling. However, there are very few research works on neural-topic models with pretrained word embedding for generating high-quality topics from short texts. In this work, in addition to pretrained word embedding, a fine-tuning stage with an original corpus is proposed for training neural-topic models in order to generate semantically coherent, corpus-specific topics. An extensive study with eight neural-topic models has been completed to check the effectiveness of additional fine-tuning and pretrained word embedding in generating interpretable topics by simulation experiments with several benchmark datasets. The extracted topics are evaluated by different metrics of topic coherence and topic diversity. We have also studied the performance of the models in classification and clustering tasks. Our study concludes that though auxiliary word embedding with a large external corpus improves the topic coherency of short texts, an additional fine-tuning stage is needed for generating more corpus-specific topics from short-text data.

With the increasing proliferation of mobile phone, internet and communication technologies, social network sites (SNS) are gaining importance worldwide. People express and exchange their opinion on various social network sites like twitter, facebook, blogs over different local and global issues. Efficient analysis of such vast text data from SNS provides a good way of understanding insights of public opinion, government policy and social condition of different countries. Topic modeling is a popular tool for extracting information from text data. Dynamic topic tracking and its visualization provides a means for capturing the change of topics over time which is important for visualization of changing needs of the society and keeping updated with the current situation. In this work, COVID-19 related twitter data in two different languages are collected and analyzed by dynamic topic model to track the spread of the evolved topics during the pandemic in two different countries in order to visualize the differences and commonness of the effect of pandemic. Here we mainly focused on the tweet data related to Japan and India in Japanese and English respectively. It is found that the country specific characteristics are prominent in some topics while some topics express the general concerns during the pandemic. This study seems to be effective to provide a technique for capturing the opinion and needs of people during a pandemic by analysis of tweet data.