We have designed a real-world smart building energy fault detection (SBFD) system on a cloud-based Databricks workspace, a high-performance computing (HPC) environment for big-data-intensive applications powered by Apache Spark. By avoiding a Smart Building Diagnostics as a Service approach and keeping a tightly centralized design, the rapid development and deployment of the cloud-based SBFD system was achieved within one calendar year. Thanks to Databricks’ built-in scheduling interface, a continuous pipeline of real-time ingestion, integration, cleaning, and analytics workflows capable of energy consumption prediction and anomaly detection was implemented and deployed in the cloud. The system currently provides fault detection in the form of predictions and anomaly detection for 96 buildings on an active military installation. The system’s various jobs all converge within 14 min on average. It facilitates the seamless interaction between our workspace and a cloud data lake storage provided for secure and automated initial ingestion of raw data provided by a third party via the Secure File Transfer Protocol (SFTP) and BLOB (Binary Large Objects) file system secure protocol drivers. With a powerful Python binding to the Apache Spark distributed computing framework, PySpark, these actions were coded into collaborative notebooks and chained into the aforementioned pipeline. The pipeline was successfully managed and configured throughout the lifetime of the project and is continuing to meet our needs in deployment. In this paper, we outline the general architecture and how it differs from previous smart building diagnostics initiatives, present details surrounding the underlying technology stack of our data pipeline, and enumerate some of the necessary configuration steps required to maintain and develop this big data analytics application in the cloud.

Predicting rehabilitation outcomes is essential for guiding clinical decisions and improving patient care. Traditional machine learning methods, while effective, are often limited in their ability to capture complex, nonlinear relationships in data. This study investigates the application of deep learning techniques, including hybrid Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), to predict rehabilitation success based on clinical and patient-reported outcome measures (CROMs and PROMs). Using a dataset of 1047 rehabilitation patients encompassing diverse musculoskeletal conditions and treatment protocols, we compare the performance of deep learning models with previously established machine learning approaches such as Random Forest and Extra Trees classifiers. Our findings reveal that deep learning significantly enhances predictive performance. The weighted F1-score for direct classification improved from 65% to 74% using a CNN-RNN architecture, and the mean absolute error (MAE) for regression-based success metrics decreased by 12%, translating to more precise estimations of functional recovery. These improvements hold clinical significance as they enhance the ability to tailor rehabilitation interventions to individual patient needs, potentially optimizing recovery timelines and resource allocation. Moreover, attention mechanisms integrated into the deep learning models provided improved interpretability, highlighting key predictors such as age, range of motion, and PROM indices. This study underscores the potential of deep learning to advance outcome prediction in rehabilitation, offering more precise and interpretable tools for clinical decision-making. Future work will explore real-time applications and the integration of multimodal data to further refine these models.