Peter Harremoës

Dr. Peter Harremoës

GSK, Niels Brock, Copenhagen, Hovedstaden, Denmark
orcid

Research Focus & Metrics

Academic Career Length

25Years (2002-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 June 2014 in IEEE Transactions on Information Theory
Verified
Journal Article
Closed access
TETim van Erven
Tim van Erven
Peter HarremoësPeter Harremoës
Peter Harremoës
Rényi divergence is related to Rényi entropy much like Kullback-Leibler divergence is related to Shannon's entropy, and comes up in many settings. It was introduced by Rényi as a measure of information that satisfies almost the same axioms as Kullback-Leibler divergence, and depends on a parameter that is called its order. In particular, the Rényi divergence of order 1 equals the Kullback-Leibler divergence. We review and extend the most important properties of Rényi divergence and Kullback-Leibler divergence, including convexity, continuity, limits of \(\sigma \) -algebras, and the relation of the special order 0 to the Gaussian dichotomy and contiguity. We also show how to generalize the Pythagorean inequality to orders different from 1, and we extend the known equivalence between channel capacity and minimax redundancy to continuous channel inputs (for all orders) and present several other minimax results.

12 May 2009 in Physical Review A
Verified
Journal Article
Closed access
PDF file
JBJop Briët
Jop Briët
Peter HarremoësPeter Harremoës
Peter Harremoës
Jensen-Shannon divergence (JD) is a symmetrized and smoothed version of the most important divergence measure of information theory, Kullback divergence. As opposed to Kullback divergence it determines in a very direct way a metric; indeed, it is the square of a metric. We consider a family of divergence measures (JDα for α>0), the Jensen divergences of order α, which generalize JD as JD1=JD. Using a result of Schoenberg, we prove that JDα is the square of a metric for α∊(0,2], and that the resulting metric space of probability distributions can be isometrically embedded in a real Hilbert space. Quantum Jensen-Shannon divergence (QJD) is a symmetrized and smoothed version of quantum relative entropy and can be extended to a family of quantum Jensen divergences of order α (QJDα). We strengthen results by Lamberti and co-workers by proving that for qubits and pure states, QJDα1/2 is a metric space which can be isometrically embedded in a real Hilbert space when α∊(0,2]. In analogy with Burbea and Rao’s generalization of JD, we also define general QJD by associating a Jensen-type quantity to any weighted family of states. Appropriate interpretations of quantities introduced are discussed and bounds are derived in terms of the total variation and trace distance.

4 January 2008 in IEEE Transactions on Information Theory
Verified
Journal Article
Closed access
Peter HarremoësPeter Harremoës
Peter Harremoës
IVIgor Vajda
Igor Vajda
This paper compares the power divergence statistics of orders with the information divergence statistic in the problem of testing the uniformity of a distribution. In this problem, the information divergence statistic is equivalent to the entropy statistic. Extending some previously established results about information diagrams, it is proved that the information divergence statistic in this problem is more efficient in the Bahadur sense than any power divergence statistic of order . This means that the entropy provides in this sense the most efficient way of characterizing the uniformity of a distribution.

20 February 2014 in IEEE Transactions on Information Theory
Verified
Journal Article
Closed access
DBDaniel Berend
Daniel Berend
Peter HarremoësPeter Harremoës
Peter Harremoës
AKAryeh Kontorovich
Aryeh Kontorovich
Pinsker's widely used inequality upper-bounds the total variation distance ∥P - Q∥ 1 in terms of the Kullback-Leibler divergence D(P∥Q). Although, in general, a bound in the reverse direction is impossible, in many applications the quantity of interest is actually D*(v, Q)-defined, for an arbitrary fixed Q, as the infimum of D(P∥Q) over all distributions P that are at least v-far away from Q in total variation. We show that D*(v, Q) ≤ Cv 2 + O(v 3 ), where C = C(Q) = 1/2 for balanced distributions, thereby providing a kind of reverse Pinsker inequality. Some of the structural results obtained in the course of the proof may be of independent interest. An application to large deviations is given.

5 December 2007 in Journal of Theoretical Probability
Verified
Journal Article
Closed access
PDF file
Peter HarremoësPeter Harremoës
Peter Harremoës
KHKlaus Holst
Klaus Holst

1 April 2009 in Entropy
Verified
Journal Article
Open access
PDF file
Peter HarremoësPeter Harremoës
Peter Harremoës
In a compact group the Haar probability measure plays the role of uniform distribution. The entropy and rate distortion theory for this uniform distribution is studied. New results and simplified proofs on convergence of convolutions on compact groups are presented and they can be formulated as entropy increases to its maximum. Information theoretic techniques and Markov chains play a crucial role. The convergence results are also formulated via rate distortion functions. The rate of convergence is shown to be exponential.