Big data has a substantial role nowadays, and its importance has significantly increased over the last decade. Big data’s biggest advantages are providing knowledge, supporting the decision-making process, and improving the use of resources, services, and infrastructures. The potential of big data increases when we apply it in real-time by providing real-time analysis, predictions, and forecasts, among many other applications. Our goal with this article is to provide a viewpoint on how to build a system capable of processing big data in real-time, performing analysis, and applying algorithms. A system should be designed to handle vast amounts of data and provide valuable knowledge through analysis and algorithms. This article explores the current approaches and how they can be used for the real-time operations and predictions.
1 March 2022 in Future Generation Computer Systems
The effective use of time series data is crucial in business decision-making. Temporal data reveals temporal trends and patterns, enabling decision-makers to make informed decisions and prevent potential problems. However, missing values in time series data can interfere with the analysis and lead to inaccurate conclusions. Thus, our work proposes a Focalize K-NN method that leverages time series properties to perform missing data imputation. This approach shows the benefits of taking advantage of correlated features and temporal lags to improve the performance of the traditional K-NN imputer. A similar approach could be employed in other methods. We tested this approach with two datasets, various parameter and feature combinations, and observed that it is beneficial in scenarios with disjoint missing patterns. Our findings demonstrate the effectiveness of Focalize K-NN for imputing missing values in time series data. The more noticeable benefits of our methods occur when there is a high percentage of missing data. However, as the amount of missing data increases, so does the error.
Every year, the amount of data produced in health-care institutions in the field of medical imaging is increasing. The easy accessibility to digital modalities and the proliferation of diagnosis centers working in network of services, creates the need for very large repositories of data. Moreover, the institutions have redundant storage across the network and load balancing mechanism to improve access performance. NoSQL databases provide horizontal scalability which allows the system to grow easily and effortlessly. Document databases are particularly suitable to store the DICOM metadata and the JSON format is already supported by the standard. This paper, we discuss the use of document database model in the medical imaging field, trying to focus on both clinical and research environments. Moreover, we focused on the proposal of an architecture based on MongoDB to distribute the load between multiple nodes. The proposal was integrated with an open-source PACS and validated by simulating multiple operations over data distributed across multiple virtual locations, and the results are promising.