Nonparametric statistical testing of EEG- and MEG-data,☆☆

https://doi.org/10.1016/j.jneumeth.2007.03.024Get rights and content

Abstract

In this paper, we show how ElectroEncephaloGraphic (EEG) and MagnetoEncephaloGraphic (MEG) data can be analyzed statistically using nonparametric techniques. Nonparametric statistical tests offer complete freedom to the user with respect to the test statistic by means of which the experimental conditions are compared. This freedom provides a straightforward way to solve the multiple comparisons problem (MCP) and it allows to incorporate biophysically motivated constraints in the test statistic, which may drastically increase the sensitivity of the statistical test. The paper is written for two audiences: (1) empirical neuroscientists looking for the most appropriate data analysis method, and (2) methodologists interested in the theoretical concepts behind nonparametric statistical tests. For the empirical neuroscientist, a large part of the paper is written in a tutorial-like fashion, enabling neuroscientists to construct their own statistical test, maximizing the sensitivity to the expected effect. And for the methodologist, it is explained why the nonparametric test is formally correct. This means that we formulate a null hypothesis (identical probability distribution in the different experimental conditions) and show that the nonparametric test controls the false alarm rate under this null hypothesis.

Introduction

The topic of this paper is the statistical analysis of ElectroEncephaloGraphic (EEG) and MagnetoEncephaloGraphic (MEG) data. These data will subsequently be denoted together as MEEG-data. MEEG-data have a spatiotemporal structure: the signal is sampled at multiple sensors and multiple time points (as determined by the sampling frequency). The data are typically collected in different experimental conditions and the experimenter wants to know if there is a difference between the data observed in these conditions. In most studies, the conditions differ with respect to the type of stimulus being presented immediately before or during the registration of the signal. In other studies, the conditions differ with respect to the type of response (e.g., correct or incorrect) that was given.

In the statistical analysis of MEEG-data we have to deal with the multiple comparisons problem (MCP). This problem originates from the fact that the effect of interest is evaluated at an extremely large number of (sensor, time)-pairs. This number is usually in the order of several thousands. The MCP involves that, due to the large number of statistical comparisons (i.e., one per (sensor, time)-pair), it is not possible to control the so-called family-wise error rate (FWER) by means of the standard statistical procedures that operate at the level of single (sensor, time)-pairs. The FWER is the probability under the hypothesis of no effect of falsely concluding that there is a difference between the experimental conditions at one or more (sensor, time)-pairs. A solution of the MCP requires a procedure that controls the FWER at some critical alpha-level (typically, 0.05 or 0.01).

In this paper, we discuss nonparametric statistical testing of MEEG-data. Contrary to the familiar parametric statistical framework, it is straightforward to solve the MCP in the nonparametric framework. Nonparametric tests were first proposed for testing the difference between MEEG-waveforms at a particular sensor (Blair and Karniski, 1993, elaborating on a parametric procedure proposed by Guthrie and Buchwald, 1991), then for MEEG-topographies at a particular time point Achim, 2001, Galán et al., 1997, Karnisky et al., 1994, and finally also for whole spatiotemporal matrices (Maris, 2004). Nonparametric tests have also been used very successfully for frequency domain representations of EEG- and MEG-data Kaiser and Lutzenberger, 2005, Kaiser et al., 2000, Kaiser et al., 2003, Kaiser et al., 2006, Lutzenberger et al., 2002. Recently, nonparametric tests were proposed for distributed inverse solutions obtained by a minimum variance beamformer Chau et al., 2004, Singh et al., 2003 or a minimum norm linear inverse (Pantazis et al., 2005). Finally, nonparametric tests for fMRI-data were proposed by Holmes et al. (1996), Bullmore et al., 1996, Bullmore et al., 1999, Nichols and Holmes (2002), Raz et al. (2003), and Hayasaka and Nichols, 2003, Hayasaka and Nichols, 2004.

The present paper contributes to the literature in several respects: (1) it explains how the sensitivity of the statistical test can be drastically improved by incorporating biophysically motivated constraints in the test statistic, (2) it is written in a tutorial-like fashion, enabling neuroscientists to construct their own statistical test, maximizing the sensitivity to the expected effect, and (3) it explains why the nonparametric test is formally correct, making use of the so-called conditioning rationale, a concept that is both rigorous and intuitive. The paper is written for two audiences: (1) empirical neuroscientists looking for the most appropriate data analysis method, and (2) methodologists interested in the theoretical concepts behind nonparametric statistical tests. With the empirical neuroscientist in mind, we have written Sections 2 Methods, 3 Results in a tutorial-like fashion, and with the methodologist in mind, we have written Section 4 that is sufficiently rigorous.

Section snippets

Methods

We make use of an example data set that was obtained in a study on the semantic processing of sentences (Jensen et al., submitted). This study involved a comparison of two experimental conditions that differed with respect to the semantic congruity of the final word in a sentence with the first part of the sentence. As is the case for most neuroscience studies, the authors are interested in the difference between experimental conditions with respect to the biological data. A central point of

Single-sensor analyses

For well-studied experimental paradigms, one often knows at which sensor the strongest effect can be observed. For instance, previous EEG-studies in which semantic congruity was manipulated (for a review, see Kutas and Federmeier, 2000) have shown the maximum effect over parietal cortex near the midline (Pz and the surrounding electrodes). And previous MEG-studies Simos et al., 1997, Helenius et al., 1998, Helenius et al., 2002, Halgren et al., 2002 have shown a dipolar pattern over left

Justification

Until now, we have deliberately ignored three important issues: (1) the exact specification of the null hypothesis that is tested by the nonparametric statistical test, (2) the proof that this test controls the FA rate, and (3) the issue of how to choose a test statistic. The theory of nonparametric statistical tests is not well documented and not very accessible. Surprisingly, the central argument in this theory (the so-called conditioning rationale, see further) is rather intuitive and can

The permutation test for multiple-subject MEEG studies

In practice, one is often interested in a null hypothesis about a population of subjects, instead of a single subject. In a very similar way as for a single-subject MEEG study, the null hypothesis about a population can be tested by means of a permutation test. To test the null hypothesis at the level of a population, a sample of subjects is drawn from this population. The first step involves taking the average2

Conclusions

We have shown how nonparametric statistical tests can be used to evaluate different effect types that are studied in the MEEG-literature (i.e., single-sensor and multi-sensor evoked responses and time–frequency representations). We have also presented a theory for these nonparametric statistical tests, which demonstrates their validity in a rigorous way. This theory applies to both single-subject and multiple-subject studies. The null hypothesis of a nonparametric statistical test involves that

References (35)

Cited by (5453)

View all citing articles on Scopus

The methods described in this paper have been implemented in the Matlab toolbox Fieldtrip, which is available from http://www.ru.nl/fcdonders/fieldtrip.

☆☆

The authors would like to thank Ole Jensen for generously sharing his data.

View full text