13 April 2021 in IEEE Access
The unstructured data impacts 95% of the organizations and costs them millions of dollars annually. If managed well, it can significantly improve business productivity. The traditional information extraction techniques are limited in their functionality, but AI-based techniques can provide a better solution. A thorough investigation of AI-based techniques for automatic information extraction from unstructured documents is missing in the literature. The purpose of this Systematic Literature Review (SLR) is to recognize, and analyze research on the techniques used for automatic information extraction from unstructured documents and to provide directions for future research. The SLR guidelines proposed by Kitchenham and Charters were adhered to conduct a literature search on various databases between 2010 and 2020. We found that: 1. The existing information extraction techniques are template-based or rule-based, 2. The existing methods lack the capability to tackle complex document layouts in real-time situations such as invoices and purchase orders, 3. The datasets available publicly are task-specific and of low quality. Hence, there is a need to develop a new dataset that reflects real-world problems. Our SLR discovered that AI-based approaches have a strong potential to extract useful information from unstructured documents automatically. However, they face certain challenges in processing multiple layouts of the unstructured documents. Our SLR brings out conceptualization of a framework for construction of high-quality unstructured documents dataset with strong data validation techniques for automated information extraction. Our SLR also reveals a need for a close association between the businesses and researchers to handle various challenges of the unstructured data analysis.
12 July 2021 in IEEE Access
The daily transaction of an organization generates a vast amount of unstructured data such as invoices and purchase orders. Managing and analyzing unstructured data is a costly affair for the organization. Unstructured data has a wealth of hidden valuable information. Extracting such insights automatically from unstructured documents can significantly increase the productivity of an organization. Thus, there is a huge demand to develop a tool that can automate the extraction of key fields from unstructured documents. Researchers have used different approaches for extracting key fields, but the lack of annotated and high-quality datasets is the biggest challenge. Existing work in this area has used standard and custom datasets for extracting key fields from unstructured documents. Still, the existing datasets face some serious challenges, such as poor-quality images, domain-related datasets, and a lack of data validation approaches to evaluate data quality. This work highlights the detailed process flow for end-to-end key fields extraction from unstructured documents. This work presents a high-quality, multi-layout unstructured invoice documents dataset assessed with a statistical data validation technique. The proposed multi-layout unstructured invoice documents dataset is highly diverse in invoice layouts to generalize key field extraction tasks for unstructured documents. The proposed multi-layout unstructured invoice documents dataset is evaluated with various feature extraction techniques such as Glove, Word2Vec, FastText, and AI approaches such as BiLSTM and BiLSTM-CRF. We also present the comparative analysis of feature extraction techniques and AI approaches on the proposed multi-layout unstructured invoice document dataset. We attained the best results with BiLSTM-CRF model.
5 January 2024 in E3S Web of Conferences
This research explores consumer behaviour towards sustainable product choices, emphasizing the influence of visual trends. Utilizing a comprehensive survey, we examined aspects such as environmental impact awareness during purchases, recycling practices, and reactions to eco-labelling. The data, visualized through Python, revealed a preference for sustainable products among environmentally aware consumers. However, a paradox emerges with the persistent preference for non-reusable plastics, despite awareness of their environmental impact. These findings underscore the complexity of consumer behaviour and highlight the necessity for education and practical measures to foster truly sustainable choices. This study contributes novel insights into the dynamics of consumer decision-making in the context of environmental sustainability.
20 March 2026 in MethodsX
Social media sites provide warning signs for shifts in consumer behavior, competitive forces, and emerging market trends. However, most small and medium-sized businesses (SMBs) do not have a systematic and scalable approach to tap into this unstructured data from various sites to extract insights. This paper proposes a trend detection method that leverages automated data extraction with n8n workflows, transformer-based embeddings, hybrid sentiment analysis, BERTopic clustering, and a weighted TrendScore composite score. The proposed approach combines multiple, heterogeneous inputs into a single analytical workflow and offers explainable visual and conversational BI interfaces, which are specifically designed for SMBs. The parameter definitions, scoring rules, and workflow diagrams are carefully detailed to ensure that the approach is fully reproducible. The proposed approach focuses on interpretability, robustness across multiple platforms, and applicability within a resource-constrained business setting. • Reproducible multi-platform social data acquisition using exportable n8n workflows. • Hybrid Transformer-Lexicon sentiment modeling combined with BERTopic clustering. • Quantified TrendScore integrating growth, engagement, sentiment shift, and cross-platform consistency.