28 December 2015 in WIREs Computational Statistics
7 March 2017 in Annual Review of Statistics and Its Application
The energy of data is the value of a real function of distances between data in metric spaces. The name energy derives from Newton's gravitational potential energy, which is also a function of distances between physical objects. One of the advantages of working with energy functions (energy statistics) is that even if the data are complex objects, such as functions or graphs, we can use their real-valued distances for inference. Other advantages are illustrated and discussed in this review. Concrete examples include energy testing for normality, energy clustering, and distance correlation. Applications include genome studies, brain studies, and astrophysics. The direct connection between energy and mind/observations/data in this review is a counterpart of the equivalence of energy and matter/mass in Einstein's E=mc2.
1 November 2009 in ASTIN Bulletin
A new approach to goodness-of-fit for Pareto distributions is introduced. Based on Euclidean distances between sample elements, the family of statistics and tests is indexed by an exponent in (0,2) on Euclidean distance. The corresponding tests are statistically consistent and have excellent performance when applied to heavy-tailed distributions. The exponent can be tailored to the particular Pareto distribution. The goodness-of-fit statistic measures all types of differences between distributions, hence it is also applicable as a minimum distance estimator. Implementation of the test statistics is developed and applied to estimation of the tail index in three well known examples of claims data, and compared with the classical EDF statistics.
1 April 2010 in Pattern Recognition
3 November 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Energy statistics was proposed by Sz\' ekely in the 80's inspired by Newton's gravitational potential in classical mechanics and it provides a model-free hypothesis test for equality of distributions. In its original form, energy statistics was formulated in Euclidean spaces. More recently, it was generalized to metric spaces of negative type. In this paper, we consider a formulation for the clustering problem using a weighted version of energy statistics in spaces of negative type. We show that this approach leads to a quadratically constrained quadratic program in the associated kernel space, establishing connections with graph partitioning problems and kernel methods in machine learning. To find local solutions of such an optimization problem, we propose kernel k-groups, which is an extension of Hartigan's method to kernel spaces. Kernel k-groups is cheaper than spectral clustering and has the same computational cost as kernel k-means (which is based on Lloyd's heuristic) but our numerical results show an improved performance, especially in higher dimensions. Moreover, we verify the efficiency of kernel k-groups in community detection in sparse stochastic block models which has fascinating applications in several areas of science.
1 July 2021 in Journal of Statistical Planning and Inference