The ever-increasing use of generative models in various fields where tabular data is used highlights the need for robust and standardized validation metrics to assess the similarity between real and synthetic data. Current methods lack a unified framework and rely on diverse and often inconclusive statistical measures. Divergences, which quantify discrepancies between data distributions, offer a promising avenue for validation. However, traditional approaches calculate divergences independently for each feature due to the complexity of joint distribution modeling. This paper addresses this challenge by proposing a novel approach that uses divergence estimation to overcome the limitations of marginal comparisons. Our core contribution lies in applying a divergence estimator to build a validation metric considering the joint distribution of real and synthetic data. We leverage a probabilistic classifier to approximate the density ratio between datasets, allowing the capture of complex relationships. We specifically calculate two divergences: the well-known Kullback-Leibler (KL) divergence and the Jensen-Shannon (JS) divergence. KL divergence offers an established use in the field, while JS divergence is symmetric and bounded, providing a reliable metric. The efficacy of this approach is demonstrated through a series of experiments with varying distribution complexities. The initial phase involves comparing estimated divergences with analytical solutions for simple distributions, setting a benchmark for accuracy. Finally, we validate our method on a real-world dataset and its corresponding synthetic counterpart, showcasing its effectiveness in practical applications. This research offers a significant contribution with applicability beyond tabular data and the potential to improve synthetic data validation in various fields.
Sentiment analysis is an active topic in Natural Language Processing (NLP). It has attracted a significant interest of research community due to the wide range of applications, including social-media, fake news spotting and interactive applications. In this paper, we present a novel approach for semi-automatic background creation and conspiracy classification. For this purpose, a complete framework including novel recurrent models is proposed. The BORJIS : Best algorithm foR Joint conspiracy and sarcasm detection has been tested on twitter-crawled data and It is composed by: $(a)$ the crawler and labelling module, $(b)$ the features vector extraction and $(c)$ the conspiracy classifier. BORJIS was compared with up-to-date techniques and it showed a significant improvement (≥ 10% accuracy) when applied to diverse datasets.
21 May 2025 in IEEE Open Journal of the Computer Society
Membership attacks pose a major issue in terms of secure machine learning, especially in cases in which real data are sensitive. Models tend to be overconfident in predicting labels from the training set. Nevertheless, its application has traditionally been limited to supervised models, while in the case of generative models we have found that there is a lack of theoretical foundations to bring this concept into the scene. Hence, this article provides the theoretical background in the context of membership inference attacks and their relationship to generative models, including the derivation of an evaluation metric. In addition, the link between these types of attack and differential privacy is shown to be a particular case. Lastly, we empirically show through simulations the intuition and application of the concepts derived.
Over the last few years, there have been several attempts to provide software tools for the development of federated learning (FL) models. However, both the complexity of the concept itself and the high entry barrier of these tools have meant that their adoption has been limited. Considering the related benefits, especially in terms of preserving data privacy, and the need for this type of solution in specific areas where data sharing is impossible, not only from a practical point of view but also from a legal and even ethical perspective, it is necessary to advance in solutions that allow its use to be democratised and its deployment to be extended. With this objective in mind, FLIP (Federated Learning Interactive Platform) has been developed as a comprehensive, easy-to-use fully functional web-based FL network management platform that eases and accelerates the usage of federated datasets by researchers in real scenarios. In this sense, FLIP has achieved a SUS score of 84.64, confirming a high level of perceived usability as expected. Taking this into account, FLIP can help increase the productivity and adoption of FL by a wider audience.
18F FDG PET/CT imaging is emerging as a tool for Multiple Myeloma (MM) evaluation. The goal of this study is to assess the most informative regions and features in MM patients using a fully automatic imaging-based pipeline by evaluating the performance of survival models with radiomics features for risk stratification. A dataset of whole-body-18F FDG PET/CT images from 227 patients was used. All images were first automatically segmented to find the spine and the remaining skeletal structures. Several masks were then generated using this segmentation, and radiomic features were extracted from both images applying these masks. The features were aggregated to create three datasets per mask: one per imaging type and a dataset combining both. Four survival model families were trained on these datasets to estimate progression-free survival. In total, 128 model × image-origin × mask configurations were tuned to maximize mean c-index via five-fold cross-validation. The best performing model that utilized both imaging modalities was then analyzed to assess the prognostic power of its features. To achieve the best ranking performance, the best configuration was a DeepSurv model trained solely with features from the PET image extracted using a mask created by using the spine and the region around and inside it. It achieved a mean c-index of 0.657 and IBS of 0.197 using 5-fold cross-validation. Image-texture features were the most relevant for risk prediction, with higher heterogeneity correlating with increased risk of an event. The pipeline is available in the project's GitLab repository. The best performing models were random survival forests and DeepSurv, which outperformed Cox-based linear models. Models with both CT and PET provided better results than with just one, on average, however the most informative imaging technique was PET, with more features present when combining both images and the overall best performing model. When building the masks, the inclusion of the paramedullary region was important, although adding the whole skeleton did not improve results. The most relevant features for survival analysis were textural features, with image heterogeneity correlating with a higher risk of disease progression.