Setup.

The first step is to create the Network object. Here we choose to provide the TPM in 2-dimensional state-by-node form (see § 2-dimensional state-by-node form). The TPM is the only required argument, but we provide the CM as well, since we know that there are no self-loops in the system and PyPhi will use this information to speed up the computation. We also label the nodes A, B, and C to make the output easier to read.

We select a subsystem and a system state for analysis by creating a Subsystem object. System states are represented by tuples of 1s and 0s, with 1 meaning “ON” and 0 meaning “OFF.” In this case we will analyze the entire system, so the subsystem will contain all three nodes. The nodes to include can be specified with either their labels or their indices (note that in other PyPhi functions, nodes must be specified with their indices).

If there are nodes outside the subsystem, they are considered as background conditions for the causal analysis [3]. In the graphical representation of the system, the background conditions are fixed in their current state while the subsystem is perturbed and observed in order to derive its TPM. In the TPM representation, the equivalent operation is performed by conditioning the system TPM on the state at t of the nodes outside the subsystem and then marginalizing out those nodes at t + 1 (illustrated in S1 Text). In PyPhi, this is done when the subsystem is created; the subsystem TPM can be accessed with the tpm attribute, e.g. subsystem.tpm.

Cause/Effect repertoires (mechanism-level information).

The lowest-level objects in the CES of a system are the cause repertoire and effect repertoire of a set of nodes within the subsystem, called a mechanism, over another set of nodes within the subsystem, called a purview of the mechanism. The cause (effect) repertoire is a probability distribution that captures the information specified by the mechanism about the purview by describing how the previous (next) state of the purview is constrained by the current state of the mechanism.

In terms of graphical operations, the effect repertoire is obtained by (1) fixing the mechanism nodes in their state at t; (2) noising the non-mechanism nodes at time t, so as to remove their causal influence on the purview; and (3) observing the resulting state transition from t to t + 1 while ignoring the state at t + 1 of non-purview nodes, in order to derive a distribution over purview states at t + 1.

The cause repertoire is obtained similarly, but in that case, the purview nodes at time t − 1 are noised, and the resulting state transition from t − 1 to t is observed while ignoring the state of non-mechanism nodes. Bayes’ rule is then applied, resulting in a distribution over purview states at t − 1. The corresponding operations on the TPM are detailed in § Calculation of cause/effect repertoires from the TPM and illustrated in S1 Text.

Note that, operationally, we enforce that each input from a noised node conveys independent noise during the perturbation/observation step. In this way, we avoid counting correlations from outside the mechanism-purview pair as constraints due to the current state of the mechanism. Graphically, this process would correspond to replacing each noised node that is a parent of multiple purview nodes (for the effect repertoire) or mechanism nodes (for the cause repertoire) with multiple, independent “virtual nodes” (as in [3, Supplementary Methods]). However, the equivalent definition of repertoires in Eqs (3) and (5) obviates the need to actually implement virtual nodes in PyPhi.

With the cause_repertoire() method of the Subsystem, we can obtain the cause repertoire of, for example, mechanism A over the purview ABC depicted in Fig. 4 of [3]:

We see that mechanism A in its current state, ON (1), specifies information by ruling out the previous states in which B and C are OFF (0). That is, the probability that either (0, 0, 0) or (1, 0, 0) was the previous state, given that A is currently ON, is zero:

Note that repertoires are returned in multidimensional form, so they can be indexed with state tuples as above. Repertoires can be reshaped to be 1-dimensional if needed, e.g. for plotting, but care must be taken that NumPy’s FORTRAN (column-major) ordering is used so that PyPhi’s little-endian convention for indexing states is respected (see § 2-dimensional state-by-node form). PyPhi provides the pyphi.distribution.flatten() function for this:

Minimum-information partitions (mechanism-level integration).

Having assessed the information of a mechanism over a purview, the next step is to assess its integrated information (denoted φ) by quantifying the extent to which the cause and effect repertoires of the mechanism-purview pair cannot be reduced to the repertoires of its parts.

In terms of graphical operations, the irreducibility of a mechanism-purview pair is tested by partitioning it into parts and cutting the connections between them. By applying the perturbation/observation procedure after cutting the connections we obtain a partitioned repertoire. Since the partition renders the parts independent, in terms of TPM manipulations, the partitioned repertoire can be calculated as the product of the individual repertoires for each of the parts. If the partitioned repertoire is no different than the original unpartitioned repertoire, then the mechanism as a whole did not specify integrated information about the purview. By contrast, if a repertoire cannot be factored in this way, then some of its selectivity is due to the causal influence of the mechanism as an integrated whole on the purview, and the repertoire is said to be irreducible.

The amount of irreducibility of a mechanism over a purview with respect to a partition is quantified as the distance between the unpartitioned repertoire and the partitioned repertoire (calculated with pyphi.distance.repertoire_distance()). There are many ways to divide the mechanism and purview into parts, so the irreducibility is measured for every partition and the partition that yields the minimum irreducibility is called the minimum-information partition (MIP). The integrated information (φ) of a mechanism-purview pair is the distance between the unpartitioned repertoire and the partitioned repertoire associated with the MIP. PyPhi supports several distance measures and partitioning schemes (see § Configuration).

The MIP search procedure is implemented by the Subsystem.cause_mip() and Subsystem.effect_mip() methods. Each returns a RepertoireIrreducibilityAnalysis object that contains the MIP, as well as the φ value, mechanism, purview, temporal direction (cause or effect), unpartitioned repertoire, and partitioned repertoire. For example, we compute the effect MIP of mechanism ABC over purview ABC from Fig. 6 of [3] as follows:

Here we can see that the MIP attempts to factor the repertoire of ABC over ABC into the product of the repertoire of ABC over AC and the repertoire of the empty mechanism ∅ over B. However, the repertoire cannot be factored in this way without information loss; the distance between the unpartitioned and partitioned repertoire is nonzero (). Thus mechanism ABC over the purview ABC is irreducible.

Maximally-irreducible cause-effect repertoires (mechanism-level exclusion).

Next, we apply IIT’s postulate of exclusion at the mechanism level by finding the maximally-irreducible cause (MIC) and maximally irreducible effect (MIE) specified by a mechanism. This is done by searching over all possible purviews for the RepertoireIrreducibilityAnalysis object with the maximal φ value. The Subsystem.mic() and Subsystem.mie() methods implement this search procedure; they return a MaximallyIrreducibleCause and a MaximallyIrreducibleEffect object, respectively. The MIC of mechanism BC, for example, is the purview AB (Fig. 8 of [3]). This is computed like so:

Concepts.

If the mechanism’s MIC has φ_cause > 0 and its MIE has φ_effect > 0, then the mechanism is said to specify a concept. The φ value of the concept as a whole is the minimum of φ_cause and φ_effect.

We can compute the concept depicted in Fig. 9 of [3] using the Subsystem.concept() method, which takes a mechanism and returns a Concept object containing the φ value, the MIC (in the ‘cause’ attribute), and the MIE (in the ‘effect’ attribute):

Note that in PyPhi, the repertoires are distributions over purview states, rather than system states. Occasionally it is more convenient to represent repertoires as distributions over the entire system. This can be done with the expand_cause_repertoire() and expand_effect_repertoire() methods of the Concept object, which assume the unconstrained (maximum-entropy) distribution over the states of non-purview nodes:

Also note that Subsystem.concept() will return a Concept object when φ = 0 even though these are not concepts, strictly speaking. For convenience, bool(concept) evaluates to True if φ > 0 and False otherwise.

Cause-effect structures (system-level information).

The next step is to compute the CES, the set of all concepts specified by the subsystem. The CES characterizes all of the causal constraints that are intrinsic to a physical system. This is implemented by the pyphi.compute.ces() function, which simply calls Subsystem.concept() for every mechanism , where is the power set of subsystem nodes. It returns a CauseEffectStructure object containing those Concepts for which φ > 0.

We see that every mechanism in except for AC specifies a concept, as described in Fig. 10 of [3]:

Irreducible cause-effect structures (system-level integration).

At this point, the irreducibility of the subsystem’s CES is evaluated by applying the integration postulate at the system level. As with integration at the mechanism level, the idea is to measure the difference made by each partition and then take the minimal value as the irreducibility of the subsystem.

We begin by performing a system cut. Graphically, the subsystem is partitioned into two parts and the edges going from one part to the other are cut, rendering them causally ineffective. This is implemented as an operation on the TPM as follows: Let E_cut denote the set of directed edges in the subsystem that are to be cut, where each edge e ∈ E_cut has a source node a and a target node b. For each edge, we modify the individual TPM of node b (Fig 2) by marginalizing over the states of a at t. The resulting TPM specifies the function implemented by b with the causal influence of a removed. We then combine the modified node TPMs to recover the full TPM of the partitioned subsystem. Finally, we recalculate the CES of the subsystem with this modified TPM (the partitioned CES).

The irreducibility of a CES with respect to a partition is the distance between the unpartitioned and partitioned CESs (calculated with pyphi.compute.ces_distance(); several distances are supported; see § Configuration). This distance is evaluated for every partition, and the minimum value across all partitions is the subsystem’s integrated information Φ, which measures the extent to which the CES specified by the subsystem is irreducible to the CES under the minimal partition.

This procedure is implemented by the pyphi.compute.sia() function, which returns a SystemIrreducibilityAnalysis object (Fig 1). We can verify that the Φ value of the example system in [3] is 1.92 and the minimal partition is that which removes the causal connections from AB to C:

Complexes (system-level exclusion).

The final step in unfolding the CES of the system is to apply the postulate of exclusion at the system level. We compute the CES of each subset of the network, considered as a subsystem (that is, fixing the external nodes as background conditions), and find the CES with maximal Φ, called the maximally-irreducible cause-effect structure (MICS) of the system. The subsystem giving rise to it is called the major complex; any overlapping subsets with lower Φ are excluded. Non-overlapping subsets may be further analyzed to find additional complexes within the system.

In this example, we find that the whole system ABC is the system’s major complex, and all proper subsets are excluded:

Note that since pyphi.compute.major_complex() is a function of the Network, rather than a particular Subsystem, it is necessary to specify the state in which the system should be analyzed.

2-dimensional state-by-node form.

A TPM in state-by-node form is a matrix where the entry (i, j) gives the probability that the j^th node will be ON at t + 1 if the system is in the i^th state at t. This representation has the advantage of being more compact than the state-by-state form, with 2ⁿ × n entries instead of 2ⁿ × 2ⁿ, where n is the number of nodes. Note that the TPM admits this representation because in PyPhi the nodes are binary; both Pr(N_t+1 = ON) and Pr(N_t+1 = OFF) can be specified by a single entry, in our case Pr(N_t+1 = ON), since the two probabilities must sum to 1.

Because the possible system states at t are represented implicitly as row indices in 2-dimensional TPMs, there must be an implicit mapping from states to indices. In PyPhi this mapping is achieved by listing the state tuples in lexicographical order and then interpreting them as binary numbers where the state of the first node corresponds to the least-significant bit, so that e.g. the state (0, 0, 0, 1) is mapped to the row with index 8 (the ninth row, since Python uses zero-based indexing [18]). Designating the first node’s state as the least-significant bit is analogous to choosing the little-endian convention in organizing computer memory. This convention is preferable because the mapping is stable under the inclusion of new nodes: including another node in a subsystem only requires concatenating new rows and a new column to its TPM rather than interleaving them. Note that this is opposite convention to that used in writing numbers in positional notation; care must be taken when converting between states and indices and between different TPM representations (the pyphi.convert module provides convenience functions for these purposes).

Multidimensional state-by-node form.

When a state-by-state TPM is provided to PyPhi by the user, it is converted to state-by-node form and the conditional independence property (Eq (1)) is checked. Note that any TPM in state-by-node form necessarily satisfies Eq (1). For internal computations, the TPM is then reshaped so that it has n + 1 dimensions rather than two: the first n dimensions correspond to the states of each of the n nodes at t, while the last dimension corresponds to the probabilities of each node being ON at t + 1. In other words, the indices of the rows (current states) in the 2-dimensional TPM are “unraveled” into n dimensions, with the i^th dimension indexed by the i^th bit of the 2-dimensional row index according to the little-endian convention. Because the TPM is stored in a NumPy array, this multidimensional form allows us to take advantage of NumPy indexing [19] and use a state tuple as an index directly, without converting it to an integer index:

The first entry of this array signifies that if the state of the system is (0, 1, 1) at t, then the probability of the first node N₀ being ON at t + 1 is Pr(N_0,t+1 = ON) = 1. Similarly, the second entry means Pr(N_1,t+1 = ON) = 0.25 and the third entry means Pr(N_2,t+1 = ON) = 0.75.

Most importantly, the multidimensional representation simplifies the calculation of marginal and conditional distributions and cause/effect repertoires, because it allows efficient “broadcasting” [19] of probabilities when multiplying distributions. Specifically, the Python multiplication operator ‘*’ acts as the tensor product when the operands are NumPy arrays A and B of equal dimensionality such that for each dimension d, either A.shape[d] == 1 or B.shape[d] == 1.

The effect repertoire.

We begin with the simplest case: calculating the effect repertoire of a mechanism M ⊆ S over a purview consisting of a single element P_i ∈ S. This is defined as a conditional probability distribution over states of the purview element at t + 1 given the current state of the mechanism, (2) It is derived from the TPM by conditioning on the state of the mechanism elements, marginalizing over the states of non-purview elements P′ = S\P_i (these states correspond to columns in the state-by-state TPM), and marginalizing over the states of non-mechanism elements M′ = S\M (these correspond to rows):

This operation is implemented in PyPhi by several subroutines. First, in a pre-processing step performed when the Subsystem object is created, a Node object is created for each element in the subsystem. Each Node contains its own individual TPM, extracted from the subsystem’s TPM; this is a 2^s × 2 matrix where s is the number of the node’s parents and the entry (i, j) gives the probability that the node is in state j (0 or 1) at t + 1 given that its parents are in state i at t. This node TPM is represented internally in multidimensional state-by-node form as usual, with singleton dimensions for those subsystem elements that are not parents of the node. The effect repertoire is then calculated by conditioning the purview node’s TPM on the state of the mechanism nodes that are also parents of the purview node, via the pyphi.tpm.condition_tpm() function, and marginalizing out non-mechanism nodes, with pyphi.tpm.marginalize_out().

In cases where there are mechanism nodes that are not parents of the purview node, the resulting array is multiplied by an array of ones that has the desired shape (dimensions of size two for each mechanism node, and singleton dimensions for each non-mechanism node). Because of NumPy’s broadcasting feature, this step is equivalent to taking the tensor product of the array with the maximum-entropy distribution over mechanism nodes that are not parents, so that the final result is a distribution over all mechanism nodes, as desired.

The effect repertoire over a purview of more than one element is given by the tensor product of the effect repertories over each individual purview element, (3) Again, because PyPhi TPMs and repertoires are represented as tensors (multidimensional arrays), with each dimension corresponding to a node, the NumPy multiplication operator between distributions over different nodes is equivalent to the tensor product. Thus the effect repertoire over an arbitrary purview is trivially implemented by taking the product of the effect repertoires over each purview node with numpy.multiply().

The cause repertoire.

The cause repertoire of a single-element mechanism M_i ∈ S over a purview P ⊆ S is defined as a conditional probability distribution over the states of the purview at t − 1 given the current state of the mechanism, (4) As with the effect repertoire, it is obtained by conditioning and marginalizing the TPM. However, because the TPM gives conditional probabilities of states at t + 1 given the state at t, Bayes’ rule is first applied to express the cause repertoire in terms of a conditional distribution over states at t − 1 given the state at t, where the marginal distribution Pr(P_t−1) over previous states is the uniform distribution. In this way, the analysis captures how a mechanism in a state constrains a purview without being biased by whether certain states arise more frequently than others in the dynamical evolution of the system [3, 4, 11, 16]. Then the cause repertoire can be calculated by marginalizing over the states of non-mechanism elements M′ = S\M_i (now corresponding to columns in the state-by-state TPM) and non-purview elements P′ = S\P (now corresponding to rows),

In PyPhi, the “backward” conditional probabilities Pr(m_i,t | P_t−1) for a single mechanism node are obtained by indexing into the last dimension of the node’s TPM with the state m_i,t and then marginalizing out non-purview nodes via pyphi.tpm.marginalize_out(). As with the effect repertoire, the resulting array is then multiplied by an array of ones with the desired shape in order to obtain a distribution over the entire purview. Finally, because in this case the probabilities were obtained from columns of the TPM, which do not necessarily sum to 1, the distribution is normalized with pyphi.distribution.normalize().

The cause repertoire of a mechanism with multiple elements is the normalized tensor product of the cause repertoires of each individual mechanism element, (5) where is a normalization factor that ensures that the distribution sums to 1. This is implemented in PyPhi via numpy.multiply() and pyphi.distribution.normalize(). For a more complete illustration of these procedures, see S1 Text.

Connectivity optimizations.

As mentioned in § Input, providing connectivity information explicitly with a CM can greatly reduce the time complexity of the computations, because in certain cases missing connections imply reducibility a priori.

For example, at the system level, if the subsystem is not strongly connected then Φ is necessarily zero. This is because a unidirectional cut between one system part and the rest can always be found that will not actually remove any edges, so the CESs with and without the cut will be identical (see S3 Text for proof). Accordingly, PyPhi immediately excludes these subsystems when finding the major complex of a system.

Similarly, at the mechanism level, PyPhi uses the CM to exclude certain purviews from consideration when computing a MIC or MIE by efficiently determining that repertoires over those purviews are reducible without needing to explicitly compute them. Suppose there are two sets of nodes X and Y for which there exist partitions X = (X₁, X₂) and Y = (Y₁, Y₂) such that there are no edges from X₁ to Y₂ and no edges from X₂ to Y₁. Then the effect repertoire of mechanism X over purview Y can be factored as and the cause repertoire of mechanism Y over purview X can be factored as Thus in these cases the mechanism is reducible for that purview and φ = 0 (see S4 Text for proof).

Analytical solution to the earth mover’s distance.

One of the repertoire distances available in PyPhi is the earth mover’s distance (EMD), with the Hamming distance as the ground metric. Computing the EMD between repertoires is a costly operation, with time complexity O(n2³ⁿ) where n is the number of nodes in the purview [21]. However, when comparing effect repertoires, PyPhi exploits a theorem that states that the EMD between two distributions p and q over multiple nodes is the sum of the EMDs between the marginal distributions over each individual node, if p and q are independent. This analytical solution has time complexity O(n), a significant improvement over the general EMD algorithm (note that this estimate does not include the cost of computing the marginals, which already have been computed to obtain the repertoires). By the conditional independence property (Eq 1), the conditions of the theorem hold for EMD calculations between effect repertoires, and thus the analytical solution can be used for half of all repertoire calculations performed in the analysis. The theorem is formally stated and proved in S5 Text.

Approximations.

Currently, two approximate methods of computing Φ are available. These can be used via settings in the PyPhi configuration file (they are disabled by default):

1. pyphi.config.CUT_ONE_APPROXIMATION (the “cut one” approximation), and
2. pyphi.config.ASSUME_CUTS_CANNOT_CREATE_NEW_CONCEPTS (the “no new concepts” approximation).

In both cases, the complexity of the calculation is greatly reduced by replacing the optimal partitioned CES by an approximate solution. The system’s Φ value is then computed as usual as the difference between the unpartitioned CES and the approximate partitioned CES.

Cut one.

The “cut one” approximation reduces the scope of the search for the MIP over possible system cuts. Instead of evaluating the partitioned CES for each of the 2ⁿ unidirectional bipartitions of the system, only those 2n bipartitions are evaluated that sever the edges from a single node to the rest of the network or vice versa. Since the goal is to find the minimal Φ value across all possible partitions, the “cut one” approximation provides an upper bound on the exact Φ value of the system.

No new concepts.

For most choices of mechanism partitioning schemes and distance measures, it is possible that the CES of the partitioned system contains concepts that are reducible in the unpartitioned system and thus not part of the unpartitioned CES. For this reason, PyPhi by default computes the partitioned CES from scratch from the partitioned TPM. Under the “no new concepts” approximation, such new concepts are ignored. Instead of repeating the entire CES computation for each system partition, which requires reevaluating all possible candidate mechanisms for irreducibility, only those mechanisms are taken into account that already specify concepts in the unpartitioned CES. In many types of systems, new concepts due to the partition are rare. Approximations using the “no new concepts” option are thus often accurate. Note, however, that this approximation provides neither a theoretical upper nor lower bound on the exact Φ value of the system.