Training Deep Learning (DL) models require large, high-quality datasets, often assembled with data from different institutions. Federated Learning (FL) has been emerging as a method for privacy-preserving pooling of datasets employing collaborative training from different institutions by iteratively globally aggregating locally trained models. One critical performance challenge of FL is operating on datasets not independently and identically distributed (non-IID) among the federation participants. Even though this fragility cannot be eliminated, it can be debunked by a suitable optimization of two hyper-parameters: layer normalization methods and collaboration frequency selection. In this work, we benchmark five different normalization layers for training Neural Networks (NNs), two families of non-IID data skew, and two datasets. Results show that Batch Normalization, widely employed for centralized DL, is not the best choice for FL, whereas Group and Layer Normalization consistently outperform Batch Normalization, with a performance gain of up to about 15 % in the most challenging non-IID scenario. Similarly, frequent model aggregation decreases convergence speed and mode quality.
1 March 2014 in European Journal of Pharmaceutical Sciences
The role of hydrogen bond acidity, i.e., the ability of chemicals to act as hydrogen bond donors (HBD), is a crucial element in pharmaceutical sciences and medicinal chemistry. It has been shown that the difference between logP values (ΔlogP) obtained in two different biphasic systems is informative of the solutes HBD properties and thus useful in the prediction of drugs human fate. In this study, we collected from literature more than 200 experimental logPtol (the logarithm of the partition coefficient P in the toluene/water system) values along with their corresponding logPoct (the logarithm of the partition coefficient P in the octanol/water system) values. The dataset was processed using a purposely-built in-house software to remove molecules that are potentially able to form IMHBs. On the remaining structures the ΔlogPoct-tol (=logPoct - logPtol) have been calculated and correlated with 82 VolSurf+descriptors through a PLS model. Finally the Block Relevance (BR) analysis has been used to group the descriptors in six easy-to-interpret blocks and to show graphically the relevance of a certain block in the PLS model. BR analysis showed that the hydrogen bond donor (HBD) properties of the solutes mainly govern ΔlogPoct-tol. This supports the use of ΔlogPoct-tol to estimate HBD properties of solutes and its role in the intramolecular hydrogen bonding (IMHB) interpretation scheme recently reported in the literature.
Decentralised Machine Learning (DML) enables collaborative machine learning without centralised input data. Federated Learning (FL) and Edge Inference are examples of DML. While tools for DML (especially FL) are starting to flourish, many are not flexible and portable enough to experiment with novel processors (e.g., RISC-V), non-fully connected network topologies, and asynchronous collaboration schemes. We overcome these limitations via a domain-specific language allowing us to map DML schemes to an underlying middleware, i.e. the FastFlow parallel programming library. We experiment with it by generating different working DML schemes on x86-64 and ARM platforms and an emerging RISC-V one. We characterise the performance and energy efficiency of the presented schemes and systems. As a byproduct, we introduce a RISC-V porting of the PyTorch framework, the first publicly available to our knowledge.
11 January 2015 in Journal of Computer-Aided Molecular Design
Lorenzo Paletto, Valerio Basile, Roberto Esposito. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.