This paper proposes a novel two-stage ensemble framework combining Long Short-Term Memory (LSTM) and Bidirectional LSTM (BiLSTM) with randomized feature selection to enhance diabetes prediction accuracy and calibration. The method first trains multiple LSTM/BiLSTM base models on dynamically sampled feature subsets to promote diversity, followed by a meta-learner that integrates predictions into a final robust output. A systematic simulation study conducted reveals that feature selection proportion critically impacts generalization: mid-range values (0.5–0.8 for LSTM; 0.6–0.8 for BiLSTM) optimize performance, while values close to 1 induce overfitting. Furthermore, real-life data evaluation on three benchmark datasets—Pima Indian Diabetes, Diabetic Retinopathy Debrecen, and Early Stage Diabetes Risk Prediction—revealed that the framework achieves state-of-the-art results, surpassing conventional (random forest, support vector machine) and recent hybrid frameworks with an accuracy of up to 100%, AUC of 99.1–100%, and superior calibration (Brier score: 0.006–0.023). Notably, the BiLSTM variant consistently outperforms unidirectional LSTM in the proposed framework, particularly in sensitivity (98.4% vs. 97.0% on retinopathy data), highlighting its strength in capturing temporal dependencies.

The analysis of high-dimensional count data presents a unique set of challenges, including overdispersion, zero-inflation, and complex nonlinear relationships that traditional generalized linear models and standard machine learning approaches often fail to adequately address. This study introduces and validates a novel Random Forest framework specifically developed for high-dimensional Poisson and Negative Binomial regression, designed to overcome the limitations of existing methods. Through comprehensive simulations and a real-world genomic application to the Norwegian Mother and Child Cohort Study, we demonstrate that the proposed methods achieve superior predictive accuracy, quantified by lower root mean squared error and deviance, and critically produced exceptionally stable and interpretable feature selections. Our theoretical and empirical results show that these distribution-optimized ensembles significantly outperform both penalized-likelihood techniques and naive-transformation-based ensembles in balancing statistical robustness with biological interpretability. The study concludes that the proposed frameworks provide a crucial methodological advancement, offering a powerful and reliable tool for extracting meaningful insights from complex count data in fields ranging from genomics to public health.

Ensemble methods have proven highly effective in enhancing predictive performance by combining multiple models. We introduce a novel ensemble approach, the Random Generalized Additive Logistic Forest (RGALF), which integrates generalized additive models (GAMs) within a random forest framework to improve binary classification tasks. Unlike traditional random forests, which rely on piecewise constant predictions in terminal nodes, RGALF fits GAM logistic regression (LR) models to the data in each terminal node, enabling it to capture complex nonlinear relationships and interactions among predictors. By aggregating these node-specific GAMs, RGALF addresses multicollinearity, enhances interpretability, and achieves superior bias–variance tradeoffs, particularly in nonlinear settings. Theoretical analysis confirms that RGALF achieves Stone’s optimal rates for additive models ($\mathcal{O}(n^{-2k/(2k+d)}$) under appropriate conditions, outperforming the slower convergence of traditional random forests ($\mathcal{O}\left(n^{-2/3}\right)$). Furthermore, empirical results demonstrate RGALF’s effectiveness across both simulated and real-world datasets. In simulations, RGALF demonstrates superior performance over random forests (RFs), reducing variance by up to 69% and bias by 19% in nonlinear settings, with significant MSE improvements (0.032 vs. RF’s 0.054 at $n=1000$), while achieving optimal convergence rates ($\mathcal{O}\left(n^{-0.48}\right)$ vs. RF’s $\mathcal{O}\left(n^{-0.29}\right)$). On real-world medical datasets, RGALF attains near-perfect accuracy and AUC: 100% accuracy/AUC for Heart Failure and Hepatitis C (HCV) prediction, 99% accuracy/100% AUC for Pima Diabetes, and 98.8% accuracy/100% AUC for Indian Liver Patient (ILPD), outperforming state-of-the-art methods. Notably, RGALF captures complex biomarker interactions (BMI–insulin in diabetes) missed by traditional models.

Fractional cointegration has been extensively examined in time series analysis, but its extension to heterogeneous panel data with unobserved heterogeneity and cross-sectional dependence remains underdeveloped. This paper develops a robust framework for testing fractional cointegration in heterogeneous panel data, where unobserved heterogeneity, cross-sectional dependence, and persistent shocks complicate traditional approaches. We propose the Bayesian Tapered Narrowband Least Squares (BTNBLS) estimator, which addresses three critical challenges: (1) spectral leakage in long-memory processes, mitigated via tapered periodograms; (2) precision loss in fractional parameter estimation, resolved through narrowband least squares; and (3) unobserved heterogeneity in cointegrating vectors (${\theta }_i$) and memory parameters ($\nu ,\delta$), modeled via hierarchical Bayesian priors. Monte Carlo simulations demonstrate that BTNBLS outperforms conventional estimators (OLS, NBLS, TNBLS), achieving minimal bias (0.041–0.256), near-nominal coverage probabilities (0.87–0.94), and robust control of Type 1 errors (0.01–0.07) under high cross-sectional dependence ($\rho =0.8$), while the Bayesian Chen–Hurvich test attains near-perfect power (up to 1.00) in finite samples. Applied to Purchasing Power Parity (PPP) in 18 fragile Sub-Saharan African economies, BTNBLS reveals statistically significant fractional cointegration between exchange rates and food price ratios in 15 countries ($p<0.05$), with a pooled estimate ($\widehat{\theta }=0.33$, $p<0.001$) indicating moderate but resilient long-run equilibrium adjustment. These results underscore the importance of Bayesian shrinkage and spectral tapering in panel cointegration analysis, offering policymakers a reliable tool to assess persistence of shocks in institutionally fragmented markets.

Longitudinal data, characterized by repeated measurements on the same subjects over time, are ubiquitous in biomedical sciences, economics, social sciences, and engineering. Analyzing such data presents unique statistical and computational challenges, including within-subject correlation, time-varying covariates, irregular observation times, informative dropout, and high dimensionality. While traditional statistical methods, such as linear mixed-effects models and generalized estimating equations, remain foundational, they often struggle with complex nonlinear dynamics, ultra-high-dimensional feature spaces, and very large sample sizes. Over the past two decades, machine learning (ML) and artificial intelligence (AI) methods have emerged as powerful complementary approaches to address these limitations. This review provides a comprehensive survey of mathematical and computational methods for longitudinal data analysis. We cover classical statistical models, penalized regression techniques, tree-based ensemble methods, kernel machines, Bayesian hierarchical models, and modern deep learning architectures, including recurrent neural networks, temporal convolutional networks, attention-based Transformers, neural ordinary differential equations, and generative models. We propose a unified taxonomy that organizes existing methods along two primary axes: the underlying mathematical framework and the analytical objective. For each category, we present detailed mathematical formulations, discuss key theoretical properties, examine computational considerations, and summarize representative reported applications drawn from the published literature. To increase the practical value of this review, we provide a cross-cutting comparison of method families against five key challenges (within-subject correlation, irregular sampling, missing data, high dimensionality, and scalability) and offer concrete guidance on method selection according to sample size, dimensionality, and analytical objective. Finally, we critically evaluate the strengths and limitations of these approaches, with particular emphasis on interpretability, scalability, handling of missing data, robustness to covariance misspecification, and uncertainty quantification.

Evaluating whether clinical classifiers perform equitably across patient subgroups is a central requirement for the responsible deployment of machine learning in medicine. Conventional approaches test one fairness metric at a time, such as sensitivity, positive predictive value, or area under the receiver operating characteristic curve, and therefore cannot detect disparities that manifest only in the joint structure of a group-specific confusion matrix. We develop a unified hypothesis-testing framework rooted in random matrix theory that compares demographic groups through the $L^2$ distance between their joint eigenvalue densities, yielding a scalar spectral divergence that is sensitive to every cell of the $2\times 2$ confusion matrix simultaneously. We derive the closed-form spectral divergence for Gaussian-approximated eigenvalue densities, prove almost-sure consistency of the empirical estimator via the delta method, and construct an extreme-value (Gumbel) test statistic with family-wise error rate control. Monte Carlo experiments comprising $\mathrm{10,000}$ replications across balanced, moderately imbalanced, and severely imbalanced group-size regimes show that the spectral test keeps Type I errors close to its nominal level while achieving power exceeding $90\%$ in complex and multi-dimensional violation scenarios, where the best single-metric competitor reaches at most $63\%$. Three clinical benchmark datasets from the UCI Machine Learning Repository utilised include Pima Indians Diabetes ($n=768$), Cleveland Heart Disease ($n=303$), and Heart Failure Clinical Records ($n=299$). Results confirm that the spectral method detects statistically significant ($p<0.001$) performance disparities missed by all three conventional tests. These results support eigenvalue-based divergence as a practical, model-agnostic diagnostic equity tool for clinical machine learning audits.