Abstract
Our sensorimotor interactions with objects are guided by their current spatial and perceptual features, as well as by learned object knowledge. A fresh red tomato is grasped differently than a soft overripe tomato, even when those objects possess the same spatial metrics of size and shape. Objects' spatial and perceptual features need to be integrated during grasping, but those features are analyzed in two anatomically distinct neural pathways. The anterior intraparietal sulcus (aIPS) might support the integration of those features. We combine transcranial magnetic stimulation (TMS) interference, EEG recordings, and psychophysical methods to test aIPS causal contributions to sensorimotor integration, characterizing the dynamics of those contributions during motor planning. Human subjects performing grasping movements were provided with visual information about a target object, namely spatial and pictorial cues, whose availability and information value were independently modulated on each trial. Maximally informative visual cues, irrespective of their spatial or perceptual nature, led to enhanced motor preparatory activity early during movement planning, and to stronger spatial congruency between finger trajectories and target object. Disturbing aIPS activity with single-pulse TMS within 200 ms after object presentation reduced those electrophysiological and behavioral indices of enhanced motor planning. TMS interference with aIPS also disturbed subjects' ability to use learned object knowledge during motor planning. These results indicate that aIPS is necessary for the fast generation of a new motor plan on the basis of both spatial and pictorial cues. Furthermore, as learned object knowledge becomes available, aIPS comes to strongly depend on this prior information for structuring the motor plan.
Introduction
When grasping a fruit, we adjust our grip to its size, but also to its perceived ripeness. When handling a tool, we adjust our hand to its shape and intended use. These examples suggest that object-guided actions rely on pictorial features and learned knowledge of the target object, in addition to its spatial properties (Passingham et al., 1998). However, spatial and pictorial information is processed along two anatomically distinct pathways in the early cerebral stages—a dorsal and a ventral stream (Felleman and Van Essen, 1991; Goodale and Milner, 1992). How are these different cerebral sources of object information integrated to guide hand–object interactions?
Recent findings suggest that the anterior intraparietal sulcus (aIPS), known to be involved in processing metric features such as size and shape to guide hand and fingers toward an object (Murata et al., 2000; Tanné-Gariépy et al., 2002; Rizzolatti and Matelli, 2003; Gamberini et al., 2009), is also in a suitable anatomical and functional position to integrate nonmetric pictorial cues and stored knowledge during motor behavior. Namely, aIPS is strongly monosynaptically connected to occipitotemporal regions involved in pictorial processing (Rozzi et al., 2006; Borra et al., 2008). Furthermore, the coupling between aIPS and the lateral occipital complex increases when pictorial information must be integrated into motor commands (Verhagen et al., 2008).
Here, we test whether aIPS is causally involved in the integration of spatial, pictorial, and learned object knowledge during the planning of grasping movements. Using single-pulse transcranial magnetic stimulation (TMS), we quantify effects of the TMS intervention by concurrently recording both electroencephalographic (EEG) and kinematic changes. This multimodal approach provided us with a broad and sensitive observation window on the consequences of the TMS intervention, from early movement preparation to late execution periods. We operationalized our research question by manipulating the sources and information value of depth cues necessary to organize an effective grasping movement. We bias subjects to estimate the object's depth either through spatial disparity or perceptual pictorial cues. We control this bias by manipulating the viewing conditions and slant of the target object (Knill, 2005). Spatial disparity cues, present with binocular object vision and processed in the dorsal stream (Georgieva et al., 2009; Minini et al., 2010), are most relevant for the estimation of vertically slanted objects. By contrast, pictorial cues such as texture and shading, processed in ventral stream (Mon-Williams et al., 2001; Georgieva et al., 2008), dominate in the perception of horizontally slanted objects. Furthermore, as subjects accumulate experience with the task, a third, more abstract source of information arises: learned knowledge of the possible object configurations and hand–object interactions.
We hypothesize that disturbing aIPS processing affects the integration of spatial, pictorial, and learned object knowledge during grasp planning as a function of the availability and relevance of those sources of information. Accordingly, the prediction is that interference with aIPS activity should disrupt subjects' ability to grasp vertically slanted objects during binocular vision, or horizontally slanted objects during monocular vision, and most strongly when object knowledge has accumulated with experience across trials.
Materials and Methods
Subjects
Thirty healthy, right-handed subjects (mean ± SD, age: 21 ± 2 years; 9 males; handedness: 89 ± 11) (Oldfield, 1971) participated in the experiment. All participants conformed to standard EEG and TMS exclusion criteria: subjects with a personal (or close familial) history of neurological or psychiatric disorders, epilepsy, migraine, or cardiac arrhythmia, were excluded from participation. Subjects had normal or corrected-to-normal vision; stereoscopic disparity threshold was at least 120 arcmin (49 ± 29 arcmin, mean ± SD) (TNO stereographs, Laméris Ootech BV). The study was approved by the local ethics committee, and written informed consent was obtained before the start of the experiment according to the Declaration of Helsinki. Two subjects were excluded from analyses because they performed the task too slowly [i.e., their mean reaction times (RTs) were >1.5 times the interquartile range (IQR) below the first quartile of the group]. Four additional subjects were excluded because of high electrode impedances, muscle artifacts, or eye blink artifacts, leaving the datasets of 24 subjects to enter the analyses.
Experimental setup
Subjects were seated at a table, with their head stabilized on a chin rest. They wore earplugs, and their right hand was resting on a button box, pressing down the button marking the starting position (home-key). They were facing two mechanical shutters that could be opened and closed independently in 3.4 ms (Fig. 1A) to control their vision. A black prism (6 × 6 × 2 cm), serving as a target object, was positioned along the midsagittal plane in front of the subject, displayed against a white background, at a comfortable reaching distance (∼25 cm in front and ∼30 cm above the starting position of the right hand). The object could be oriented in different slants around its pitch axis, in seven steps between the frontal and transversal plane relative to the subject. In the vertical orientation, the object was upright, with the object's largest surface (6 × 6 cm) parallel to the subject's frontal plane. With increasing slant, the top of the object moved away from the subject, toward a horizontal orientation parallel to the table surface. It spanned a visual angle of ∼7° when oriented vertically (Figs. 1A, 2A).
Experimental task
Each trial started with subjects holding their right hand on the home-key, their vision occluded by closed shutters. After a random time interval (2–4 s), in which the object was rotated to one of seven slants (0:15:90° relative to the frontal plane), either one or both shutters opened, allowing monocular or binocular vision of the object to be grasped. As soon as subjects started to grasp the object (i.e., they left the home-key), their vision was occluded. Subjects grasped the object with their right hand, removed it from its support, held it in mid-air, placed it back into its support, and returned to press the home-key (Fig. 1B). Subjects were instructed to perform the whole task at natural speed. The absence of visual feedback during movement execution, and the requirement to hold the object against gravity, further ensured that the subjects prepared an accurate grasping movement.
A trial lasted between 6 and 11 s (average duration: ∼8 s). Subjects performed four blocks of 84 trials each (∼11 min per block), for a total of 336 trials for the whole experiment (∼60 min including short breaks between each block). Two additional blocks (with 2 × 84 = 168 trials) were also collected in the same cohort of subjects; these data were obtained for the purpose of another study.
Experimental design
We considered three experimental factors: (1) TMS site (two levels: control, aIPS); (2) object slant (two levels: vertical, 0°, 15°, 30°; horizontal, 60°, 75°, 90°); and (3) object vision (two levels: binocular, monocular) (Fig. 1C). This experimental design tests for biases in visual information processing through the interaction of object slant and vision, and the effects that TMS over aIPS evokes over those biases. This approach is based on the fact that binocular disparity cues are most informative when objects are vertically slanted (compared with when they are horizontally slanted) (Knill, 2005). In contrast, monocular pictorial cues are most informative for discriminating horizontally slanted objects (Knill, 1998; Knill and Saunders, 2003). More precisely, the reliability of stereoscopic disparity cues is expected to change as the cosine of object slant from vertical to horizontal (the clockwise derivative of the horizontal projection of the slant), while the reliability of pictorial cues changes as the sine of slant (the clockwise derivative of its vertical projection). The seven object slants were binned in two factorial levels of object slant (vertical and horizontal) to comply with two analytical and statistical requirements: first, to maximize the number of trials per analytical cell; and second, as described below, the nonparametric permutation tests we performed to analyze the electrophysiological data allow only the direct comparison of two levels per experimental factor. Accordingly, we hypothesized that object slant identification (and thus the grasp plan) is most accurate when objects are vertically slanted and binocular vision is provided (binocular-vertical, relying on stereoscopic cues) or when objects are horizontally presented under monocular viewing conditions (monocular-horizontal, relying on pictorial cues).
We arbitrarily chose to provide monocular vision only to the right eye. We did not mix left and right eye vision to keep the set of monocular trials homogeneous. To prevent a preference for the right eye, also in binocular trials, we included left eye monocular trials as catch trials (∼8% of the total). In summary, each cell of the 2 × 2 × 2 design contained 33 trials; a total of 264 trials were included in the analysis, and 72 additional trials were considered catch trials (44 trials with the object slanted at 45°, 28 trials allowing only monocular vision of the left eye).
In post hoc analyses, we considered two additional factors, describing the trial interval during which the TMS intervention was delivered [factor TMS time, with two levels: early (100–200 ms) and late (300–400 ms)], and the experiment epoch during which the outcome measures were collected [factor epoch, with two levels: first half of the experiment (blocks 1 and 2), second half of the experiment (blocks 3 and 4)]. Experimental conditions were pseudorandomized and evenly distributed across trials.
Experimental intervention: transcranial magnetic stimulation
TMS pulses were delivered through a figure-eight coil (70 mm diameter), connected to a Magstim MonoPulse machine. On each trial, we delivered a single, monophasic, low-intensity TMS pulse at a randomized time within two temporal windows (early: 100–200 ms after trial onset; late: 300–400 ms after trial onset). During each of the four blocks of trials, TMS was applied over either the site of interest (aIPS) or a site controlling for nonspecific TMS effects (the vertex of the head). The order of TMS sites was counterbalanced within and between subjects: in half of the subjects the two TMS sites were ordered over the four blocks in an A-B-B-A order; in the other half, in a B-A-A-B order. The pulses were delivered at a particularly low intensity (at 100% of the subject's individual active motor threshold). In the following paragraphs, we will further elaborate on the anatomical localization and intensity of stimulation, discussing both the employed protocols and consequences for functional specificity of the perturbation by TMS.
Continuous on-line stereotactic guidance of the TMS coil was incompatible with the experimental setup. Therefore, we mapped the aIPS site off-line with respect to the 10–10 system for EEG electrode positioning in six additional reference subjects before we acquired the dataset used in this study (Jasper, 1958; Chatrian et al., 1985). We transformed the stereotactic coordinates of aIPS (Verhagen et al., 2008) into the native space of each reference subject as defined by their structural MRI scan (employing an iterative unified normalization and segmentation procedure in SPM5, Statistical Parametric Mapping; http://www.fil.ion.ucl.ac.uk/spm) (Ashburner and Friston, 2005). Using a stereotactic neuronavigation system (Brainsight, Rogue Research), the TMS coil was positioned and oriented according to the subject-specific aIPS coordinates (Fig. 1D). The average configuration of the coil for optimal aIPS stimulation was marked on the EEG caps that were subsequently used in the main experiment; the center of the coil fell close to electrode CP3. This off-line procedure proved to be robust as the positioning of the coil did not deviate >7 mm across the six additional reference subjects, i.e., within the expected spatial range of the effective magnetic field changes by the TMS pulse. The vertex site, serving as a control site, was defined as the point where both the sagittal midline from nasion to inion and the coronal line from ear to ear were dissected in the middle (Fig. 1D).
All TMS pulses were delivered at 100% of the subject's individual active motor threshold. During active motor threshold determination, subjects rested their right hand on a pillow, squeezing a small roll of tape using a pincer grip. They kept their right first dorsal interosseous (FDI) muscle continuously contracted at 15% of their individual maximum, as measured with standard electromyography. The active motor threshold was defined as the minimum stimulation intensity over contralateral motor cortex that elicited a motor-evoked potential in the FDI muscle >200 μV peak to peak in at least 5 of 10 successive stimulations (Rossini et al., 1994). Average active motor threshold was 33 ± 7% (mean ± SD) of maximum stimulator output.
We chose to stimulate at this low intensity for three reasons: first, to minimize the somatic and auditory stimulation of the subject; second, to minimize any TMS artifacts in electroencephalographic and kinematic recordings (see Electroencephalography, Data analysis, below for details); and third, and most importantly, by lowering the intensity we increased the effective functional specificity of the perturbation by the TMS pulse. When stimulating at the active motor threshold, the effectiveness of the pulse is critically dependent on the activity of the underlying cortex. The neural processes of an area that is endogenously activated by the task are more strongly perturbed than those of an area that is not engaged by the task (when at a similar distance from the coil). In this study, we target the coil at the scalp projected coordinates of a group activation in aIPS elicited during a similar task (Verhagen et al., 2008). Although the location of the intended target functional region in aIPS might differ from subject to subject, the increased functional specificity of the TMS perturbation by adopting a low intensity could be expected to improve the consistency of the functional interference of the TMS pulse across subjects.
Electroencephalography
Data acquisition.
We recorded the EEG using 31 Ag/AgCl electrodes, organized on a flat-tip cap (BrainProducts) according to the 10–10 system (Chatrian et al., 1985). Electrical voltage was sampled at 5000 Hz using amplifiers with a high dynamic range and capable of direct current recording (MR+ DC BrainAmp, BrainProducts). Before digitization, the EEG data were low-pass filtered at 1000 Hz. Electrical artifacts associated with the TMS pulse were minimized by the use of tip electrodes (rather than typical circular electrodes), and by positioning the EEG electrodes and cables perpendicular to the orientation of the TMS coil. Furthermore, to minimize the temporal spread of TMS-induced artifacts, no high-pass filter was applied at acquisition.
Data analysis.
After excluding trials with prominent eye movements, blinks, and muscle artifacts (on the basis of visual inspection of the data), EEG data were analyzed using a MATLAB toolbox (FieldTrip, http://fieldtrip.fcdonders.nl/) (Oostenveld et al., 2011) following a multistep procedure. First, the signal of each sensor was screened (in a 1 s window following the TMS pulse) for the following types of TMS-induced artifacts: (1) saturation of the EEG signal (∼2 ms after the TMS pulse), with consequent ringing artifacts (∼7 ms after the TMS pulse) due to the 1000 Hz low-pass filter (we discarded data from the time window between −0.2 and +10 ms from the TMS pulse); and (2) a slow step response (∼60–400 ms after the TMS pulse) due to step responses in the resistor–capacitor circuits and operational amplifier of the amplifier hardware, and possibly due to polarization of the electrode–electrolyte circuit (Litvak et al., 2007; Ilmoniemi and Kicić, 2010). These step responses were fitted and removed using an iterative least-squares fitting algorithm (implemented in FieldTrip) when possible; otherwise, the trial was discarded.
On average, 79 ± 6% (mean ± SD) of all trials survived the exclusion criteria and entered further analysis. Surviving data were rereferenced to the average signal of all sensors to remove any spatial effects on voltage differences with respect to the localization of the reference electrode. Data then entered into an independent component analysis (Lee et al., 1999; Makeig et al., 2004) to identify and remove residual signals related to eye movements, blinks, muscle tension, and TMS artifacts (Korhonen et al., 2011). After power-line noise was removed (using a discrete Fourier transform notch filter at 50, 100, and 150 Hz), data were bandpass filtered (0.75–150 Hz, sixth-order Butterworth filter) and downsampled to 500 Hz.
In a further analysis of the data, for each trial we calculated time–frequency representations (TFRs) of spectral power. We used a Fourier transform approach (8–35 Hz, in steps of 1 Hz) applied to sliding time windows (200 ms, sliding in 20 ms steps) multiplied by a Hanning taper (resulting in a frequency smoothing of 5 Hz). We opted to use a relatively short sliding time window (200 ms instead of, e.g., 400 ms) to enhance our temporal resolution and obtain a reliable power estimate as close to the TMS pulse as possible. Power could not be estimated in a 220 ms window around the TMS pulse (the width of one Hanning taper and one additional step: 200 + 20 ms). However, because of the trial-to-trial variation in the time of the TMS pulse, we could still generate a continuous average estimate of power. For each experimental condition, power estimates were averaged over trials, log transformed, and related to a baseline period from the same trials (relative change from −700 to −200 ms before trial onset). Individual effects were grand averaged over subjects to estimate the group effects.
Statistical inference.
Within each subject, the difference between conditions was quantified as the difference of the relative log transformed mean power changes. Statistical inference (p < 0.05) was performed at the group level (within-subjects random-effects analysis) using a nonparametric randomization test controlling for multiple comparisons over the large search space given by multiple sensors, frequency bands, and time intervals (SFT points) (Maris and Oostenveld, 2007). This procedure involved the following steps. First, the conditions of interest are described for each subject independently by the average of the corresponding trials, resulting in a summary statistic for each condition in each subject. Second, all SFT points are identified for which the t statistics for the difference between conditions over subjects exceed a threshold (paired dependent-samples t test, p > 0.05). Third, contiguous SFT points exceeding the threshold are grouped in clusters, the t values from each SFT point of a cluster are added, and this cumulative t value is used for inferential statistics at the cluster level. Fourth, a Monte Carlo estimate of the permutation p value of the cluster is obtained by comparing the cluster-level test statistic to a randomization null distribution assuming no difference between conditions. This distribution is obtained by randomly swapping the conditions within subjects and repeating the steps above to calculate the cluster-level test statistic multiple times. Using 10,000 random draws, the Monte Carlo p value is an accurate estimate of the true p value.
Hand kinematics
Data acquisition.
We sampled the position and orientation of four sensors at 250 Hz, using an electromagnetic tracking system (LIBERTY, Polhemus). Three sensors were positioned on the subject's hand: on the nail of the thumb, on the nail of the index finger, and on top of the first metacarpophalangeal joint (MCPJ) (Fig. 2A). The fourth sensor was positioned on the object to be grasped, along the axis of rotation of the prism.
Data analysis.
Kinematic data were analyzed using MATLAB (Mathworks), as follows. We first removed the TMS-induced artifact (20 ms window following the TMS pulse) and interpolated the missing values based on a piecewise cubic Hermite polynomial. The resulting data were low-pass filtered at 15 Hz using a sixth-order Butterworth filter. An additional virtual sensor (referred to as “grip sensor”) was defined as the mean of the thumb and index sensors. We then calculated velocity and acceleration of the virtual and real sensors (Fig. 2C). We computed grip aperture as the Euclidian distance between the sensors placed on the thumb and index finger (Fig. 2D), and grip velocity as the first derivative of this grip aperture (Fig. 2D). Grip orientation was defined as the angle along the midsagittal plane between a vertical vector pointing upward and the thumb–index finger axis (Fig. 2B).
We also calculated a series of parameters describing the temporal evolution of the grasping movement, to gather information on the quality of motor planning. Given that the target presented during the trial informed subjects only on the required end state of the movement, and given that visual feedback was withheld during movement execution, we focused the kinematic analyses on the accuracy of the movement just before the object is grasped between index finger and thumb. Accordingly, we distinguished the approach phase from the preceding transport phase by detecting the transition from a ballistic to a nonballistic trajectory in the hand movement (as described by the MCPJ sensor) (Fig. 2, transport: blue, approach: red). The ballistic transport phase is characterized by an acceleration phase leading to the peak velocity (PV), followed by a continuous deceleration of the MCPJ sensor. Subsequent reaccelerations of the hand are included in the approach phase. These small corrective movements, arising for instance when one finger touches the object before the other, can lead to small changes in grip velocity. Given the complex characteristics of the performed movement (Fig. 2A), we took particular care to use a robust estimate of the onset and offset of the different phases of the movement, by combining multiple sources of information (Schot et al., 2010). Movement onset was defined as the point in time with the lowest thumb and index finger velocity, smallest grip aperture, and largest distance from the object, just after the velocity of the thumb and index finger has increased above 0.02 m/s along the midsagittal plane while still in proximity of the home-key. Approach phase onset was defined as the first point in time when the velocity of the MCPJ sensor reaches a minimum after it has decreased and stays below 0.3 m/s until movement offset. Movement offset was defined as the point in time with the lowest thumb, index finger, and grip velocity, and the smallest distance between the thumb, index finger, and object, when the fingers are within 7 cm of the center of mass of the object, and the grip velocity has decreased and stays below 0.05 m/s.
We described the grasping movements by commonly used kinematic parameters: RT, movement time (MT), mean velocity (MV), PV, relative time to PV as a fraction of MT (rtPV), maximum grip aperture (MGA), and relative time to MGA as a fraction of MT (rtMGA) (Jeannerod, 1984). When applicable, these parameters were determined for the transport and approach phases separately (denoted by the prefix “T” and “A,” respectively). We also considered three additional parameters to capture planning-related features of the grip configuration during the approach phase: (1) approach grip aperture (AGA), the grip aperture at the start of the approach phase; (2) differential approach grip rotation (ΔAGR), the change in grip orientation from the start of the approach phase to 100 ms after movement offset; and (3) the integral of the approach grip velocity (∑AGV), taken over the whole approach phase. The ∑AGV is a summary descriptive of the grip velocity during the approach phase. When the grip is closing (i.e., the grip velocity is negative) continuously during the approach phase, the ∑AGV will be negative. If the grip needs to be adjusted and reopened (i.e., grip velocity is positive) before finally grasping the object, the ∑AGV will become less negative, possibly even positive. Hence, the ∑AGV parameter is a comprehensive description of the closing phase of the grip, including any potential adjustments, during the approach phase.
A full list of the kinematic parameters we considered is displayed in Table 1.
Statistical inference.
We excluded trials where subjects started moving before or at the time of the TMS pulse and trials where the main kinematic parameters deviated from the first or third quartile by more than three IQRs. On average, 87 ± 6% (mean ± SD) of all trials survived the exclusion criteria and entered further analysis.
Statistical inference of the kinematic measurements was drawn using the SPSS 16.0 software package. All parameters were checked on skewness and kurtosis, and, if found deviating, were log transformed to commit to the assumption of normal distribution (this was applicable for the trajectory length and MV in the approach phase: ATL and AMV, respectively; Tables 1, 2). Trials were averaged for each experimental condition, and the resulting means were entered in a univariate repeated-measures ANOVA testing for effects between conditions within subjects. The three parameters describing the approach phase (AGA, ∑AGV, and ΔAGR) were also entered in a within-subjects, multivariate, repeated-measures multivariate ANOVA to account for potential dependencies.
Results
In this study, we tested whether aIPS is causally involved in the integration of spatial, pictorial, and learned object knowledge during the planning of grasping movements. We operationalized this hypothesis by manipulating vision and slant of the target object, thus biasing the subject's reliance on either spatial or pictorial depth cues to specify the grasp plan (Knill, 2005). In the following sections, we first report the effects evoked by changing the slant and the vision of the target object. Second, we report the effects evoked by the interaction of these two sources of visual information. Third, we report how the effects of this interaction change as subjects improve their knowledge of the possible object configurations.
The effects of the experimental manipulations were assessed by using cerebral and behavioral parameters, namely, EEG spectral power in the alpha and beta bands, and the kinematics of the approach phase of the hand movement. We first describe the EEG results and then the kinematic results. We start each section by describing these parameters as obtained during control TMS intervention, and then we provide a description of how these parameters change when aIPS is perturbed with single-pulse TMS.
Movement planning (EEG): without aIPS perturbation
As the shutters opened and the subjects could see the object and prepare their movement, ongoing cerebral oscillations were strongly suppressed in the alpha band (8–12 Hz; p < 0.001; strongest effect over parieto-occipital electrodes) (Fig. 3B), and in the beta band (18–24 Hz; p < 0.001; strongest effect over left frontoparietal electrodes) (Fig. 3A). The TFRs of occipital electrodes O1, Oz, and O2 [Fig. 3D (Fig. 3B, identified with stars] indicate that the occipital alpha suppression covered the whole duration of the motor-planning phase, returning to slightly above baseline after movement onset and concurrent shutter closure (Fig. 3B,D). The beta power suppression recorded at electrode C3 [Fig. 3C (Fig. 3A, identified with a diamond)] was sustained during both motor-planning and execution phases (Fig. 3A,C). These observations reproduce well known effects linking alpha and beta suppression to increased computational load over visual and motor areas, respectively (Hari and Salmelin, 1997; Jensen et al., 2005). The TFR derived from occipital electrodes time locked to stimulus presentation (Fig. 3D, left) reveals an additional burst of synchronization in high beta frequencies, caused by visually evoked potentials phase locked to the opening of the shutters.
Increasing object slant enhances alpha suppression
There were strong modulations of task-related oscillatory activity as a function of object slant. Increasing object slant from vertical to horizontal led to a stronger alpha suppression (8–10 Hz) over medial parietal and frontal regions. This differential effect started late during movement planning, adding to the overall task-related sustained alpha suppression, and continued during movement execution (p = 0.004). There were no significant differential effects between trials performed with monocular or binocular vision on oscillatory activity in the alpha and beta bands.
Informative spatial and pictorial cues enhance beta suppression
Spatial depth cues were more informative when grasping vertically oriented objects under binocular vision (i.e., with reliable stereoscopic cues) than under monocular vision. In contrast, pictorial depth cues were more informative when grasping horizontally oriented objects under monocular vision (i.e., with reliable pictorial cues) than under binocular vision. When contrasting trails with more informative spatial and pictorial depth cues to those with less informative cues, there was enhanced power suppression in the beta band (18–24 Hz) over electrode C3 (“beta suppression”; p = 0.002) (Fig. 4A, left). This enhancement of beta suppression was spatially localized over the left motor cortex (Fig. 4A, left) and occurred early in the movement planning (220–400 ms after stimulus presentation) (Fig. 4B, left). This finding provides a clear neurophysiological correlate for the notion that subjects integrate both stereoscopic and pictorial information as a function of the reliability of these depth cues when planning a grasping movement. The following sections will illustrate how this enhanced beta suppression is modulated by the other experimental factors, i.e., learned object knowledge and TMS intervention over aIPS.
Learned object knowledge enhances beta suppression
The enhanced beta suppression described in the previous section was stronger in the second half of the experiment than in the first (p = 0.013) (Fig. 4D). This finding suggests that, as subjects become familiar with the possible configurations of the grasped object across trials, task experience is stored in association with the specific object configurations. When in a following trial a familiar object configuration is identified, this stored knowledge is retrieved and comes to play a significant role in shaping the motor plan.
Movement planning (EEG): following perturbation of aIPS by TMS
In this section, we focus on EEG differences between aIPS and control stimulation. To make sure that these differences are about the consequences of TMS alone, we consider here only trial epochs that followed a TMS pulse (Fig. 3C,D, central panels, dashed lines). There were no effects of aIPS stimulation over task-related oscillatory activity (main effect of site: Fig. 3C,D, central panels; gray: control; magenta: aIPS; black: aIPS-control contrast) or differences driven by the manipulation of viewing conditions (main effect of vision). Furthermore, a post hoc analysis on the consequences of delivering the TMS pulse either early (100–200 ms after stimulus presentation) or late (300–400 ms) did not reveal any differential effects either.
In the following sections, we focus on the main hypothesis of this study, the prediction that aIPS is causally involved in the integration of spatial, pictorial, and learned object knowledge during the planning of grasping movements. We test this hypothesis by quantifying the effects of aIPS stimulation on the enhanced beta suppression observed when informative spatial and pictorial depth cues are integrated in the motor plan.
aIPS perturbation prevents spatial and pictorial cues to enhance beta suppression
The enhanced beta suppression was disrupted following aIPS perturbation (p = 0.009) (Fig. 4A, right). This disruption occurred at the same location (Fig. 4A), frequency band (18–24 Hz), and time (260–460 ms after stimulus) (Fig. 4B) as the effect observed after TMS over the vertex (see section Informative spatial and pictorial cues enhance beta suppression). This effect was a consequence of TMS pulses delivered early after stimulus presentation (100–200 ms), since methodological constraints of oscillatory power estimation (see Materials and Methods, Electroencephalography, Data analysis) prevented data from trials with late TMS (300–400 ms) to be included in the average signal contributing to the effect (i.e., in the time window 260–460 ms after stimulus).
aIPS perturbation prevents learned object knowledge to enhance beta suppression
In the previous section, we reported how aIPS perturbation leads to a disruption of the enhanced beta suppression observed when informative spatial and pictorial cues are integrated into a motor plan. Here we report that this disruption was stronger in the second half of the experiment (p = 0.018). Figure 4D shows that, in the first half of the experiment, aIPS stimulation merely cancels the potential advantage of incorporating reliable depth information into the motor plan. Conversely, in the second half, when task experience has accumulated, motor planning is actually hampered when otherwise maximally informative visual cues are available and aIPS is disturbed. This interaction was tested as a post hoc analysis of the effect of the factor epoch on the TMS site × slant × vision interaction. As described in the previous section, only data from trials with early TMS contributed to this effect.
Movement execution (kinematics): without aIPS perturbation
In the following sections, we report the effects of the experimental conditions on task performance, as assessed with the kinematics of the subjects' movements. We focus on the approach phase of the grasping movement, an index of planning accuracy (see Materials and Methods, Hand kinematics, Data analysis).
Main effects of object slant and vision
As the slant of the target object increased from vertical to horizontal, the subject's hand needed to travel over a longer path. The kinematic measurements clearly isolated this basic effect (Tables 1, 2; Fig. 5). Subjects had longer MTs (Fig. 5A), reached a higher peak velocity relatively earlier in the movement (PV, rtPV) (Fig. 5B), while the maximum grip aperture occurred relatively later for more horizontally slanted objects (rtMGA). Grip orientation at the onset of the approach phase closely matched the object's vertical orientations, whereas it was rotated further forward during the approach phase in trials when the object was horizontally oriented (ΔAGR) (Fig. 6C). In line with previous reports (Mamassian, 1997; van Bergen et al., 2007), the maximum and approach grip aperture became smaller as an object's slant increased (MGA, AGA) (Figs. 5C, 6A). In contrast to these widespread effects of object slant on grasping behavior, increasing object slant did not influence the duration of movement planning (i.e., the interval between stimulus presentation and movement onset, RT: F(1,22) = 0.26, p = 0.615; Table 2). Planning the grasping movement when only monocular cues were available led to slightly shorter reaction times, possibly a consequence of the additional time required to converge on the object when binocular vision was available. The shorter reaction time evoked during monocular trials resulted in larger planning uncertainties in the transport phase of the movement, as illustrated by longer movement duration, longer traveled path, and reduced MV (Tables 1, 2; Fig. 5A).
Informative spatial and pictorial cues enhance grasping accuracy
Neither the duration of the transport and approach phases nor early markers of planning accuracy, like maximum grip aperture, were affected by the presence of informative spatial and pictorial depth cues (Fig. 5C). However, the availability of these cues influenced kinematic parameters that describe the spatial congruency between the fingers and object configuration during the approach phase (F(1,22) = 4.70, p = 0.012 on a multivariate test across approach parameters on the slant × vision interaction; Table 1). More precisely, during trials providing maximally informative stereoscopic or pictorial cues (binocular-vertical and monocular-horizontal), the fingers closed faster and further, and the grip orientation rotated more when approaching the object (Fig. 6B,C). In contrast, during trials providing less informative cues (binocular-horizontal and monocular-vertical), subjects had a smaller grip aperture at the onset of the approach phase, and their fingers closed more slowly with a smaller grip adjustment (Fig. 6A–C). These observations pertain to planned movement parameters and provide a behavioral counterpart to the enhanced planning-related activity revealed by the neurophysiological data (see Informative spatial and pictorial cues enhance beta suppression) (Fig. 4).
Movement execution (kinematics): following aIPS perturbation by TMS
aIPS stimulation had strong deleterious effects on several kinematic parameters specifically related to the approach phase of the movement. It removed the increased spatial accuracy of the approaching fingers observed in binocular-vertical and monocular-horizontal trials during the TMS control session (F(1,22) = 3.65, p = 0.030 on a multivariate test across approach parameters on the TMS site × slant × vision interaction) (Fig. 6D–F). More precisely, TMS over aIPS diminished the faster/further closing of the fingers, as well as the stronger rotational adjustment of the grip orientation during the approach phase. Together, these findings indicate that aIPS stimulation removed the relative motor-planning advantages seen in those trials where reliable depth cues were available, providing a behavioral counterpart to the TMS-induced disruption of planning-related activity revealed by the neurophysiological data. aIPS perturbation prevents spatial and pictorial cues to enhance grasping accuracy.
Discussion
We have investigated the cortical dynamics supporting the planning of grasping movements, testing the causal role of the anterior intraparietal sulcus in the integration of spatial, pictorial, and learned object features into a motor plan. We manipulated the reliance of the motor system on either visuospatially processed stereoscopic disparity cues or perceptually processed pictorial depth cues. We exploited the fact that their information value to estimate the orientation of a grasped object is differentially dependent on the object's slant (Knill, 2005). When a motor plan could be based on maximally informative depth cues, either spatial or pictorial in nature, there was enhanced suppression of oscillatory power in the beta band over motor cortex (beta suppression), at the onset of the planning phase of the movement. This enhanced beta suppression, a well known neurophysiological hallmark of motor preparation (Brown, 2007), was accompanied by behavioral indices of enhanced planning accuracy. We found improved congruency of the finger trajectories in relation to the object's configuration just before contact. These findings indicate that the grasp plan was sensitive to the reliability of the available depth cues, and that both spatial and pictorial cues were effectively integrated into the motor plan. We hypothesized that aIPS supports this integrative mechanism. Perturbation of aIPS by single-pulse TMS reduced beta suppression over motor cortex at the onset of movement planning, specifically when maximally informative depth cues were available. This reduced beta suppression was accompanied by behavioral indices of reduced planning accuracy, namely, reduced spatial consistency of the grip during the final phase of the grasping movement. These observations indicate that aIPS is causally responsible for the integration of spatial and pictorial depth cues in the motor system, and that this integration occurs early during planning.
We also considered whether this integrative mechanism is part of a general ability of the aIPS to incorporate different sources of object information (“priors”) (Körding and Wolpert, 2004) into a motor plan. Therefore, we examined a third, more abstract prior, namely, knowledge of the possible object configurations as developed over the course of the experiment. As this knowledge accumulated, beta suppression was enhanced early during planning. When aIPS was disturbed, this electrophysiological effect was reversed, even when informative depth cues were available. This finding suggests that aIPS also incorporates learned knowledge of object configurations into the motor plan and, in fact, favors it over current sensory evidence.
aIPS incorporates both spatial and pictorial depth cues into a grasp plan
It is known that the aIPS region is involved in controlling finger movements according to intrinsic object metrics during visually guided grasping (Tunik et al., 2005; Culham and Valyear, 2006; Davare et al., 2010; Sakaguchi et al., 2010). The present kinematic and electrophysiological data confirm those observations, providing novel evidence on the characteristics of aIPS contributions. We manipulated the sources and information value of depth cues necessary to organize an effective grasping movement, requiring subjects to estimate object's slant either through spatially processed stereoscopic disparity cues or perceptually processed pictorial cues. Following informative spatial or pictorial depth cues (i.e., when grasping vertical objects viewed binocularly, or horizontal objects viewed monocularly) beta suppression was enhanced over motor cortex contralateral to the grasping hand. Building on this finding, we observed that disruption of neuronal processing in aIPS caused a robust and specific reduction of this beta suppression. Given that beta suppression over motor cortex is an index of its computational load (Hari and Salmelin, 1997; Jensen et al., 2005; Neuper et al., 2006), these observations indicate that disturbing aIPS during the planning phase of a grasping movement reduces the ability of the motor system to adequately organize the movement. Moreover, because subjects grasped the object without visual feedback, the functional scope of this planning impairment can be captured by the behavioral consequences of the TMS intervention. We found that disturbing aIPS selectively impaired subjects' ability to adjust their grasping movements to the orientation of the object, leaving early phases of the movement execution unaffected. These behavioral and electrophysiological findings extend previous observations (Verhagen et al., 2008), showing that aIPS is necessary for integrating both spatial and pictorial information into a grasp plan.
aIPS provides an initial motor plan for grasping
This study further advances the understanding of the functional contribution of aIPS during grasp by investigating its temporal dynamics. There are two main findings. First, motor planning was altered (as indexed by reduced beta suppression) only when aIPS was stimulated early (<200 ms after stimulus onset). Second, the electrophysiological consequences of that perturbation were also early (260–460 ms after stimulus onset). These findings suggest that the computations implemented in aIPS are necessary to develop the initial structure of the grasping movement, in line with recent neurophysiological data from macaque anterior intraparietal area (Srivastava et al., 2009; Sakaguchi et al., 2010). These observations also fit with other TMS studies showing that aIPS is necessary to rapidly adjust prehension when the intrinsic object's characteristics change (Tunik et al., 2005; Rice et al., 2006; Taubert et al., 2010). However, the current findings suggest a reinterpretation of the role of aIPS in on-line control, emphasizing the fast generation of a new grasp plan, rather than continuous refinement of an existing plan.
aIPS incorporates learned knowledge of object configurations into a grasp plan
Previous work has described how aIPS conveys current object properties to the motor cortex (Davare et al., 2010). Here we have assessed whether, in addition to spatial and pictorial cues, aIPS is also involved in conveying more abstract sources of information to the motor system. We considered whether subjects made use of the knowledge of the object's configuration accumulated across trials. Although there were no detectable changes in subjects' behavior, and the stimuli remained the same, there was increased beta suppression in the second half of the experiment when maximally informative cues were available with control stimulation. Crucially, aIPS stimulation had a stronger effect on visuomotor integration in the second half of the experiment, even preventing the subjects from using currently available informative visual evidence. As above, this effect occurred only when aIPS was stimulated early (<200 ms after stimulus onset). This finding suggests that, as object knowledge becomes accessible, the sensorimotor system comes to strongly depend on learned knowledge, which acts as a prior probability to structure the movement plan. This interpretation fits with impairments observed in patients with ideomotor apraxia, i.e., patients with left lateralized parietofrontal lesions, often including aIPS (Haaland et al., 2000; Buxbaum et al., 2005). Although apraxia patients adjust their transport kinematics and preshape their hand according to the metrics of a target object, their prehension is often functionally inappropriate (Sirigu et al., 1995; Buxbaum et al., 2003). For example, they might skillfully grasp a spoon, but at the wrong end. Paradoxically, these patients are even more impaired when grasping familiar than nonfamiliar objects (Dawson et al., 2010). Similarly, in the current study the preparatory activity of the subjects is even more disturbed by aIPS stimulation after subjects had become familiar with the properties of the grasped object. In both cases, currently available sensory evidence appears overridden by previously acquired priors. This suggests that aIPS builds an initial motor plan by retrieving stored priors associated with an identified object configuration.
Interpretational issues
It might be argued that as monocularly derived pictorial cues remain available during binocular vision, grasping horizontal objects with binocular or monocular vision should not differ. Yet, the kinematic data indicate that subjects were more accurate when grasping horizontally slanted objects under monocular than binocular vision. This effect fits with the known preferential processing of binocular cues during grasping (Servos et al., 1992; Jackson et al., 1997; Bradshaw et al., 2004), extending those observations to a situation when binocular cues are suboptimal for the identification of object slant.
It should be emphasized that, rather than using TMS to create “virtual lesions” (Walsh and Cowey, 2000), our aim was to perturb the sensorimotor system within its physiological range. Reassuringly, we did not observe any main effects of stimulation site on oscillatory power or kinematic behavior. In fact, the TMS-driven oscillatory changes were higher-order interactions between experimental factors, restricted to the beta band over left motor cortex, and temporally remote from the TMS intervention. However, it remains to be investigated whether these observed effects are a property of aIPS exclusively (as contrasted against vertex stimulation) or are shared across a dorsolateral parietofrontal network (including ventral premotor areas).
Conclusion
In this study, we qualified and extended the contributions of aIPS during grasping movements beyond spatial visuomotor processing and on-line control. We showed that aIPS contributes to the initial planning stage, incorporating both spatial and pictorial evidence into a motor plan, and even stored knowledge of familiar object configurations. We propose that aIPS rapidly builds a motor plan based on these multiple sources of information. These findings open the way for understanding why aIPS lesions can affect apparently disparate functions like grasping (Binkofski et al., 1998), action organization (Castiello, 2005), on-line motor control (Tunik et al., 2005), and tool use (Johnson-Frey, 2004; Buxbaum et al., 2005; Frey, 2008; Dawson et al., 2010).
Footnotes
This work was supported by The Netherlands Organization for Scientific Research Open Competition Grant 400-04-379 awarded to H.C.D. and I.T.; Vici Grant 453-08-002 awarded to I.T.; and Vici Grant 453-10-003 awarded to H.C.D. We thank Inge Leunissen for her assistance with data acquisition, and Ole Jensen for his advice on methodological issues.
The authors declare no competing financial interests.
- Correspondence should be addressed to Lennart Verhagen, PO Box 9101, 6500 HB, Nijmegen, The Netherlands. lennart.verhagen{at}donders.ru.nl
This article is freely available online through the J Neurosci Open Choice option.