Elsevier

NeuroImage

Volume 118, September 2015, Pages 651-661
NeuroImage

The (in)stability of functional brain network measures across thresholds

https://doi.org/10.1016/j.neuroimage.2015.05.046Get rights and content

Highlights

  • Network measures were found to be unstable across absolute thresholds.

  • For instance, the direction of significant group differences can reverse across thresholds.

  • Network measures were found to be more stable across proportional thresholds.

  • Caution should be used when applying thresholds to functional connectivity data.

Abstract

The large-scale organization of the brain has features of complex networks that can be quantified using network measures from graph theory. However, many network measures were designed to be calculated on binary graphs, whereas functional brain organization is typically inferred from a continuous measure of correlations in temporal signal between brain regions. Thresholding is a necessary step to use binary graphs derived from functional connectivity data. However, there is no current consensus on what threshold to use, and network measures and group contrasts may be unstable across thresholds. Nevertheless, whole-brain network analyses are being applied widely with findings typically reported at an arbitrary threshold or range of thresholds. This study sought to evaluate the stability of network measures across thresholds in a large resting state functional connectivity dataset. Network measures were evaluated across absolute (correlation-based) and proportional (sparsity-based) thresholds, and compared between sex and age groups. Overall, network measures were found to be unstable across absolute thresholds. For example, the direction of group differences in a given network measure may change depending on the threshold. Network measures were found to be more stable across proportional thresholds. These results demonstrate that caution should be used when applying thresholds to functional connectivity data and when interpreting results from binary graph models.

Introduction

The human brain is a large-scale system of functionally connected brain regions. This system can be modeled as a network, or graph, by dividing the brain into a set of regions, or “nodes,” and quantifying the strength of the connections between nodes, or “edges,” as the temporal correlation in their patterns of activity. Network analysis, a part of graph theory, provides a set of summary statistics that can be used to describe complex brain networks using a reduced number of observations that can be meaningfully compared between groups and/or related to behavior (Rubinov and Sporns, 2010). Recent studies have used this approach to characterize network properties as they relate to sex (Tian et al., 2011), age (Meunier et al., 2009), and cognition (Dosenbach et al., 2007), as well as in conditions such as addiction (Chanraud et al., 2011), Alzheimer's disease (Supekar et al., 2008), schizophrenia (Alexander-Bloch et al., 2013, Bassett et al., 2008), and others (Bassett and Bullmore, 2006).

Network analyses of functional connectivity data are commonly based on the blood oxygen level-dependent (BOLD) signal in functional magnetic resonance imaging (fMRI), but can also be derived from electroencephalography (EEG) or magnetoencephalography (MEG). For example, network analysis of fMRI data was used to show that more efficient global information processing (characteristic path length) in resting state was related to high intelligence quotient (IQ; van den Heuvel et al., 2009). Another study used network analysis of MEG data to show that functional integration (characteristic path length) and functional segregation (clustering coefficient) were decreased in Alzheimer's disease (Stam et al., 2009).

However, a central challenge of applying network analysis to functional connectivity data is that many network measures were designed to be calculated on binary graphs in which connections are either present or not, whereas temporal correlations in fMRI signal are continuous from − 1 to 1. The typical approach is to: (1) define n non-overlapping nodes across the brain using an anatomical atlas or a functional parcellation method where nodes have voxels with similar time courses; (2) estimate the network by computing the entries of the n-by-n matrix representing the functional connections between node pairs, either by linear association such as correlation or by some other nonlinear measure such as mutual information; and (3) apply a threshold to produce an n-by-n binary adjacency matrix representing the network edges and to remove weak connections (Bullmore and Sporns, 2009, Simpson et al., 2013). Thus a threshold is commonly applied to construct a binary graph from functional connectivity data.

The use of thresholded binary graphs is attractive because it facilitates the calculation of many network measures and reduces the computational burden of analyzing the graph. An alternative approach is to use weighted graphs that are not binary but instead allow edges to carry some sort of continuous weight value. With weighted graphs, all edges remain in the graph and any node is connected to every other node. However, some have argued that weighted graphs are not reasonable biological structures because brain regions have sparse anatomical connections only to other specific brain regions (Sporns, 2011). Weighted graphs are also less computationally efficient, especially in the analysis of large-scale networks such as voxel-based functional connectivity networks (Telesford et al., 2011); the overwhelming number of connections makes it difficult to extract meaningful information (Serrano et al., 2009).

To define binary graphs, the applied threshold is typically absolute (correlation-based) or sparsity-based (proportional). Each approach has advantages and disadvantages. Absolute thresholds set a value for the correlation coefficient between node pairs, above which they are considered connected and below which they are not. Proportional thresholds utilize a set percentage of the strongest connections (edges), such as the top 10% of correlation values in the network. For group comparisons, using a proportional threshold ensures that the networks in each group have the same number of nodes, or network size, and the same number of edges, or global degree. This allows for more meaningful comparisons of other network measures that rely on degree. However, proportional thresholds do not take into account absolute differences in correlation values, therefore information about overall group differences may be lost. Absolute thresholds retain this information, but may result in networks with different size or degree, or in a network that is connected in one group but disconnected in the other group. While a difference in node-connectedness between groups may be informative, it confounds the comparison of graph measures that vary significantly with degree (Alexander-Bloch et al., 2010). Moreover, absolute thresholds may be too large for low-average connectivity networks or too small for high-average connectivity networks, thus eliminating strong and significant connections or overemphasizing weak connections (van Wijk et al., 2010).

There is also no consensus in the literature as to what specific threshold should be used. A large range of absolute thresholds have been applied, from a correlation coefficient of r = 0.1 (Buckner et al., 2009, supplement) to r = 0.8 (Tomasi and Volkow, 2010, supplement). Likewise a range of proportional thresholds have been reported, from 5 to 40% (e.g., Fornito et al., 2010). In attempts to show that results are not sensitive to the choice of threshold, findings are often reported across a narrow range of thresholds (e.g., Cole et al., 2013, top 2–10%; van den Heuvel et al., 2009, r = 0.3–0.5). However, this can lead to incomplete results or even misleading results if network properties are unstable across a larger range of thresholds, for example if there is a reversal of group differences in a network measure across thresholds (e.g., Scheinost et al., 2012). Lastly, even if a canonical threshold was determined and agreed upon, different preprocessing decisions such as whether to use global signal regression (GSR) may shift the distribution of correlations (Murphy et al., 2009), leading to binary graphs that are not comparable across studies.

Therefore, this study sought to characterize the (in)stability of network measures across thresholds. A large resting-state fMRI dataset was used to measure network properties across the full range of absolute and proportional thresholds. In addition, the effects of GSR on network measures across thresholds were tested in order to highlight how preprocessing decisions can influence the (in)stability of network measures.

Section snippets

Participants

One hundred right-handed individuals participated in the study. All participants provided written informed consent in accordance with the Yale Human Investigations Committee at the Yale School of Medicine. The analyses included 50 males (age 35 ± 10 years) and 50 females (age 34 ± 12 years). 99 of these were from a prior dataset of 103 subjects (Scheinost et al., 2015), 4 of whom were not included because they did not complete all 8 runs, and 1 additional participant was scanned in order to have

Absolute threshold

The survival curve for L has an inverted V-shape without GSR, with the maximum shifted to a lower threshold with GSR (Figs. 1A–F). L was longer for females or males (depending on threshold) with and without GSR (Figs. 1A–B). L did not differ by age without GSR; with GSR, L was longer for younger participants and was negatively correlated with age (Fig. 1C–D, Inline Supplementary Figure S1). The shape of the survival curve for L was consistent across runs, though an effect of run was found with

Discussion

Network measures were found to be unstable in a number of ways, particularly across absolute thresholds. Few of the survival curves were monotonic across absolute thresholds (e.g., Figs. 1A–F); this can make it difficult to parameterize the curve with a simple function (as in Scheinost et al., 2012). Most of the network measures showed ‘sign reversals’ in the direction of group differences across absolute thresholds (Fig. 4). Finally, the number of nodes showing a significant group difference

Conclusion

Network analyses are useful for summarizing large-scale brain organization, to relate features of network topology to behavior and cognition, and to examine changes in these features in clinical populations such as individuals with neurological and psychiatric disorders. However, caution must be used when conducting and interpreting network analyses. As studies work to elucidate the organization of functional brain networks in these and other contexts, the choice of thresholding approach must

Acknowledgments

We thank our research participants for their time and efforts. This work was funded by grants from the National Institutes of Health, National Institute on Drug Abuse (to KAG: K12DA00167; to DS: T32DA022975); the American Heart Association (to KAG: 14CRP18200010); and the National Science Foundation (to EF: Graduate Research Fellowship).

References (43)

  • R.G. Wise et al.

    Resting fluctuations in arterial carbon dioxide induce significant low frequency variations in BOLD signal

    NeuroImage

    (2004)
  • C.G. Yan et al.

    A comprehensive assessment of regional variation in the impact of head micromovements on functional connectomics

    NeuroImage

    (2013)
  • J. Zhang et al.

    Disrupted brain connectivity networks in drug-naive, first-episode major depressive disorder

    Biol. Psychiatry

    (2011)
  • S. Achard et al.

    Efficiency and cost of economical brain functional networks

    PLoS Comput. Biol.

    (2007)
  • A.F. Alexander-Bloch et al.

    Disrupted modularity and local connectivity of brain functional networks in childhood-onset schizophrenia

    Front. Syst. Neurosci.

    (2010)
  • A.F. Alexander-Bloch et al.

    The anatomical distance of functional connections predicts brain network topology in health and schizophrenia

    Cereb. Cortex

    (2013)
  • D.S. Bassett et al.

    Small-world brain networks

    Neuroscientist

    (2006)
  • D.S. Bassett et al.

    Hierarchical organization of human cortical networks in health and schizophrenia

    J. Neurosci.

    (2008)
  • R.L. Buckner et al.

    Cortical hubs revealed by intrinsic functional connectivity: mapping, assessment of stability, and relation to Alzheimer's disease

    J. Neurosci.

    (2009)
  • R.L. Buckner et al.

    Opportunities and limitations of intrinsic functional connectivity MRI

    Nat. Neurosci.

    (2013)
  • E. Bullmore et al.

    Complex brain networks: graph theoretical analysis of structural and functional systems

    Nat. Rev. Neurosci.

    (2009)
  • Cited by (0)

    View full text