The Gaussianity assumption in Variational AutoEncoders (VAEs) enhances computational efficiency and provides a solid theoretical basis for estimating probability distributions. However, we have empirically found that approximating distributions with non-smooth densities using the Gaussian VAE is challenging. Therefore, we propose an approach for distributional learning in VAEs that extends to estimating the quantile function while accommodating both smooth and non-smooth densities. This is achieved by utilizing the continuous ranked probability score, a strictly proper scoring rule, as our reconstruction loss. Our method can be seen as a specialized form of a nonparametric M-estimator for estimating general quantile functions, and we establish a theoretical connection between our model and quantile estimation. Furthermore, we demonstrate that our reconstruction loss functions as the lower bound of an infinite mixture of asymmetric Laplace distributions, which allows our synthetic data generation mechanism to maintain differential privacy. We validate the effectiveness of our model in capturing the underlying distribution through experiments involving synthetic data generation on real-world tabular datasets, showing that the level of data privacy can be easily adjusted.

In recent times, slicing methods have yielded a successful outcome in generative models for image, sound, and text data, primarily focusing on joint distributional learning. However, we have identified a critical limitation of the slicing approach for tabular data: it struggles to capture marginal distributional patterns, which are significant for effective tabular data synthesis. To tackle this problem, we introduce a new measure of discrepancy, the mixture Cramer–Wold distance. This measure enables us to capture both marginal and joint distributional patterns simultaneously, striking a balance between the two aspects, and we provide theoretical foundations for its application. Leveraging the power of the mixture Cramer–Wold distance, we present CWDAE (Cramer–Wold Distributional AutoEncoder), a generative model that demonstrates notable performance in generating synthetic tabular data. Furthermore, our model offers the flexibility to adjust the level of data privacy to meet specific needs easily.