The analysis of massive databases is a key issue for most applications today and the use of parallel computing techniques is one of the suitable approaches for that. Apache Spark is a widely employed tool within this context, aiming at processing large amounts of data in a distributed way. For the Statistics community, R is one of the preferred tools. Despite its growth in the last years, it still has limitations for processing large volumes of data in single local machines. In general, the data analysis community has difficulty to handle a massive amount of data on local machines, often requiring high-performance computing servers. One way to perform statistical analyzes over massive databases is combining both tools (Spark and R) via the sparklyr package, which allows for an R application to use Spark. This paper presents an analysis of Brazilian public data from the Bolsa Família Programme (BFP—conditional cash transfer), comprising a large data set with 1.26 billion observations. Our goal was to understand how this social program acts in different cities, as well as to identify potentially important variables reflecting its utilization rate. Statistical modeling was performed using random forest to predict the utilization rated of BFP. Variable selection was performed through a recent method based on the importance and interpretation of variables in the random forest model. Among the 89 variables initially considered, the final model presented a high predictive performance capacity with 17 selected variables, as well as indicated high importance of some variables for the observed utilization rate in income, education, job informality, and inactive youth, namely: family income, education, occupation and density of people in the homes. In this work, using a local machine, we highlighted the potential of aggregating Spark and R for analysis of a large database of 111.6 GB. This can serve as proof of concept or reference for other similar works within the Statistics community, as well as our case study can provide important evidence for further analysis of this important social support programme.

Deep Learning models typically require large datasets to achieve high performance, which can be challenging when applied to medical data due to the difficulty in acquiring sufficient observations, particularly when the data is imbalanced between categories like pathological and normal cases. To address this, approaches like transfer learning, ensemble methods, and data augmentation through Generative Adversarial Networks (GANs) can be utilized to improve classification tasks on small datasets. In situations with imbalanced data, resampling techniques are also crucial. This research combines transfer learning with resampling methods to boost the prediction accuracy of minority class samples in small, imbalanced datasets. Additionally, techniques like data retracing and GAN-based augmentation are applied. The dataset used includes small, imbalanced images of skin cancer, aimed at classifying them as malignant or benign. The findings reveal that models using resampling techniques achieve better results, while those without resampling underperform. This underscores the benefit of resampling in enhancing prediction, particularly for the minority class. Furthermore, using GANs for data augmentation improves model performance over those that do not incorporate this technique.