With the advancement in intelligent devices, social media, and the Internet of Things, staggering amounts of new data are being generated and the pace is continuously accelerating. Real-time analytics (RTA) has emerged as a distinct branch of big data analytics focusing on the velocity aspect of big data, in which data is prepared, processed and analyzed as it arrives, with the aim of generating insights and creating business value in near real-time. The objective of this paper is to provide an overview of key concepts and architectural approaches for designing RTA solutions, including the relevant infrastructure, processing and analytics platforms, as well as analytics techniques and tools with the most up-to-date machine learning and artificial intelligence considerations, and position these in the context of the most prominent platforms and analytics techniques. The paper develops a logical analytics stack to support the description of key functionality and relationships between relevant components in RTA solutions based on a thorough literature review and industrial practice. This provides practitioners with guidance in selecting the most appropriate solutions for their RTA problems, including the application of emerging AI technologies in this context. The paper discusses the complex event processing technology that has influenced many recent data streaming solutions in the analytics stack, and highlights the integration of machine learning and artificial intelligence into RTA solutions. Some real-life application scenarios in the finance and health domains are presented, including several of the authors’ earlier contributions, to demonstrate the utilization of the techniques and technologies discussed in this paper. Future research directions and remaining challenges are discussed.

Financial time series prediction, whether for classification or regression, has been a heated research topic over the last decade. While traditional machine learning algorithms have experienced mediocre results, deep learning has largely contributed to the elevation of the prediction performance. Currently, the most up-to-date review of advanced machine learning techniques for financial time series prediction is still lacking, making it challenging for finance domain experts and relevant practitioners to determine which model potentially performs better, what techniques and components are involved, and how the model can be designed and implemented. This review article provides an overview of techniques, components and frameworks for financial time series prediction, with an emphasis on state-of-the-art deep learning models in the literature from 2015 to 2023, including standalone models like convolutional neural networks (CNN) that are capable of extracting spatial dependencies within data, and long short-term memory (LSTM) that is designed for handling temporal dependencies; and hybrid models integrating CNN, LSTM, attention mechanism (AM) and other techniques. For illustration and comparison purposes, models proposed in recent studies are mapped to relevant elements of a generalized framework comprised of input, output, feature extraction, prediction, and related processes. Among the state-of-the-art models, hybrid models like CNN-LSTM and CNN-LSTM-AM in general have been reported superior in performance to stand-alone models like the CNN-only model. Some remaining challenges have been discussed, including non-friendliness for finance domain experts, delayed prediction, domain knowledge negligence, lack of standards, and inability of real-time and high-frequency predictions. The principal contributions of this paper are to provide a one-stop guide for both academia and industry to review, compare and summarize technologies and recent advances in this area, to facilitate smooth and informed implementation, and to highlight future research directions.

Depression, a pervasive global mental disorder, profoundly impacts daily lives. Despite numerous deep learning studies focused on depression detection through speech analysis, the shortage of annotated bulk samples hampers the development of effective models. In response to this challenge, our research introduces a transfer learning approach for detecting depression in speech, aiming to overcome constraints imposed by limited resources. In the context of feature representation, we obtain depression-related features by fine-tuning wav2vec 2.0. By integrating 1D-CNN and attention pooling structures, we generate advanced features at the segment level, thereby enhancing the model's capability to capture temporal relationships within audio frames. In the realm of prediction results, we integrate LSTM and self-attention mechanisms. This incorporation assigns greater weights to segments associated with depression, thereby augmenting the model's discernment of depression-related information. The experimental results indicate that our model has achieved impressive F1 scores, reaching 79% on the DAIC-WOZ dataset and 90.53% on the CMDC dataset. It outperforms recent baseline models in the field of speech-based depression detection. This provides a promising solution for effective depression detection in low-resource environments.

News impact analysis has become a common task conducted by finance researchers, which involves reading and selecting news articles based on themes and sentiments, pairing news events and relevant stocks, and measuring the impact of selected news on stock prices. To facilitate more efficient news selection, topic modeling can be applied to generate topics out of a large number of news documents. However, there is very limited existing literature comparing topic models in the context of finance-related news impact analysis. In this paper, we compare three state-of-the-art topic models, namely Latent Dirichlet allocation (LDA), Top2Vec, and BERTopic, in a defined scenario of news impact analysis on financial markets, where 38,240 news articles with an average length of 590 words are analyzed. A service-oriented framework for news impact analysis called “News Impact Analysis” (NIA) is advocated to leverage multiple topic models and provide an automated and seamless news impact analysis process for finance researchers. Experimental results have shown that BERTopic performed best in this scenario, with minimal data preprocessing, the highest coherence score, the best interpretability, and reasonable computing time. In addition, a finance researcher was able to conduct the entire news impact analysis process, which validated the feasibility and usability of the NIA framework.

In recent years, the number of people suffering from depression has gradually increased, and early detection is of great significance for the well-being of the public. However, the current methods for detecting depression are relatively limited, typically relying on the self-rating depression scale (SDS) and interviews. These methods are influenced by subjective or environmental factors. To improve the objectivity and efficiency of diagnosis, deep learning techniques have been applied to the field of automatic depression detection (ADD), providing a more accurate and objective approach. During interviews, transcribed interview data is one of the most commonly used modalities in ADD. However, previous studies have only utilized response texts or selected question–answer pairs, resulting in information redundancy and loss. This paper is the first to apply the multiple instance learning (MIL) framework to the field of textual interview data, aiming to overcome issues of inadequate text representation and ineffective information extraction in long texts. In the MIL framework, each instance undergoes an independent feature extraction process, ensuring that the local features of each instance are fully captured. This not only enhances the overall text representation capability but also alleviates the issue of sample imbalance in the dataset. Additionally, this paper improves upon previous aggregation strategies by introducing two hyper-parameters to accommodate the uncertainties in the field of text sentiment. An ensemble model of MT5 and RoBERTa (referred to as multi-MTRB) was constructed to extract features from each instance and output confidence scores indicating the presence of depressive information in the instances. Due to the unique design of the MIL framework, the proposed method is highly interpretable and is able to identify specific sentences that identify people from depressed patients, while introducing LIME techniques to provide more in-depth interpretation of negative instance sentences. This provides a promising approach for depression detection in the context of text interview data patterns. We evaluated the proposed method on DAIC-WOZ and E-DAIC datasets with excellent results. The F1 score is 0.88 on the DAIC-WOZ dataset and 0.86 on the E-DAIC dataset.

In the fast-evolving artificial intelligence era, the intersection of natural language processing and financial analysis has attracted significant attention, primarily due to its potential to provide valuable insights into financial market behavior. Sentiment analysis of financial news articles is a crucial aspect of this intersection, providing cues about market sentiment that may affect stock price dynamics. Traditional sentiment analysis methods often rely on rules or machine learning algorithms trained on labeled datasets, but these methods face challenges in capturing the context within the text. This paper proposes a framework that incorporates prompt engineering strategies, including a novel Domain Knowledge Chain-of-Thought (DK-CoT) strategy, integrating domain-specific financial knowledge with chain-of-thought reasoning, designed to leverage and enhance the performance of large language models (LLMs) in financial news sentiment analysis. DK-CoT has been compared with various prompt engineering techniques, including zero-shot, few-shot, and chain-of-thought, as well as other benchmark models like BERT and RoBERTa. Through comprehensive experiments and evaluations, we introduce the weighted F1 score as a more practical metric, emphasizing the disproportionate impact of negative news on financial markets, which better reflects real-world financial dynamics, as negative sentiments often lead to more significant market reactions than positive or neutral sentiments. Experimental results have shown that DK-CoT adopted in an LLM called GLM is effective in improving the performance and reliability of financial news sentiment analysis. Our findings provide insights into optimal prompt designs and highlight the importance of incorporating financial knowledge to uplift LLM performance while reducing the need for extensive computational resources and fine-tuning.