Bivariate measure of redundant information

Malte Harder, Christoph Salge, and Daniel Polani
Phys. Rev. E 87, 012130 – Published 23 January 2013

Abstract

We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But, in contrast to mutual information, redundant information denotes information that is shared about the outcome of a third variable. Formalizing this concept, and being able to measure it, is required for the non-negative decomposition of mutual information into redundant and synergistic information. Previous attempts to formalize redundant or synergistic information struggle to capture some desired properties. We introduce a new formalism for redundant information and prove that it satisfies all the properties necessary outlined in earlier work, as well as an additional criterion that we propose to be necessary to capture redundancy. We also demonstrate the behavior of this new measure for several examples, compare it to previous measures, and apply it to the decomposition of transfer entropy.

  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
9 More
  • Received 26 July 2012

DOI:https://doi.org/10.1103/PhysRevE.87.012130

©2013 American Physical Society

Authors & Affiliations

Malte Harder*, Christoph Salge, and Daniel Polani

  • Adaptive Systems Research Group University of Hertfordshire, University of Hertfordshire, AL10 9AB Hatfield, United Kingdom

  • *m.harder@herts.ac.uk
  • c.salge@herts.ac.uk
  • d.polani@herts.ac.uk

Article Text (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand
Issue

Vol. 87, Iss. 1 — January 2013

Reuse & Permissions
Access Options
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review E

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×