Article Figures & Data
Figures
- Figure 1.
Data collection scheme for research sessions. A, Experimental setup (adapted from Rastogi et al., 2020). Participants had two 96-channel microelectrode arrays placed chronically in motor cortex, which recorded neural activity while participants completed a force task. TC and SBP features were extracted from these recordings. Figure 1A is reprinted by permission from Springer Nature as indicated in the Terms and Conditions of a Creative Commons Attribution 4.0 International license (https://www.nature.com/srep/). B, Research session architecture. Each session consisted of 12–21 blocks, each of which contained ∼20 trials (see Table 1). In each trial, participants attempted to generate one of three visually-cued forces with one of four grasps: power, closed pinch, open pinch, ring pinch. During session 5, participant T8 also attempted force production using elbow extension. Each trial contained a preparatory (prep) phase, a go phase where forces were actively embodied, and a stop phase where neural activity was allowed to return to baseline. Participants were prompted with both audio and visual cues, in which a researcher squeezed or lifted an object associated with each force level. During pinch blocks, the researcher squeezed the pinchable objects (cotton ball, eraser, nasal aspirator tip) using the particular pinch grip dictated by the block (ring pinch, open pinch, closed pinch). Here, only closed pinches of objects are shown.
- Figure 2.
Exemplary TC and SBP features tuned to task parameters of interest in participant T8 (TC and SBP features in participant T5 are illustrated in Extended Data Fig. 2-1). Rows indicate average per-condition activity (PSTH) of four exemplary features tuned to force, grasp, both factors, and an interaction between force and grasp, recorded during session 5 from participant T8 (two-way Welch-ANOVA, corrected p < 0.05, Benjamini–Hochberg method). Bolded, starred p values indicate significant tuning to force (Rows 1 and 3), grasp (Rows 2 and 3), or a force-grasp interaction (Row 4). Neural activity was normalized by subtracting block-specific mean feature activity within each recording block, and then smoothed with a 100-ms Gaussian kernel to aid in visualization. Column 1 contains PSTHs averaged within individual force levels (across multiple grasps), such that observable differences between data traces are because of force alone. Similarly, column 2 shows PSTHs averaged within individual grasps (across multiple forces). Column 3 shows a graphical representation of the simple main effects as normalized mean neural deviations from baseline activity during force trials within each of the five grasps. (cp, c-pinch = closed pinch; op, o-pinch = open pinch; rp, r-pinch = ring pinch, pow = power, elb = elbow extension). Mean neural deviations were computed over the go phase of each trial and subsequently averaged within each force-grasp pair. Error bars indicate 95% confidence intervals.
- Figure 3.
Summary of neural feature population tuning to force and grasp. Row 1, Fraction of neural features significantly tuned to force, grasp, both force, and grasp and an interaction between force and grasp in participants T8 and T5 (two-way Welch-ANOVA, corrected p < 0.05). Row 2, Fraction of neural features significantly tuned to an interaction between force and grasp, subdivided into force-tuned features within each individual grasp (c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch). Note that the number of grasp types differed between sessions (see Table 1).
- Figure 4.
Simulated models of independent and interacting (grasp-dependent) neural representations of force. Row 1, Equations corresponding to the independent and interacting models of force representation. Here, xij represents neural feature activity generated during a particular grasp i and force j, gi represents baseline feature activity during grasp i, f represents force-related neural feature activity, and sj is a discrete force level. Row 2, Simulated population neural activity projected into a two-dimensional PCA space. Estimated force axes within the low-dimensional spaces are shown as blue lines. Row 3, Summary of variances accounted for by the top 20 dPCs extracted from the simulated neural data from each model. Here, the variance of each individual component is separated by marginalization (force, grasp, and interaction between force and grasp). Pie charts indicate the percentage of total signal variance due to these marginalizations.
- Figure 5.
Neural population-level activity patterns. A, Demixed principal components (dPCs) isolated from all force-grasp conditions from T8 session 5, all force-grasp conditions from T5 session 7, and power versus elbow conditions from T8 session 5 neural data. dPCs were tuned to four marginalizations of interest: Condition-Independent (CI) tuning (i.e., time), Force, Grasp, and an interaction between force and grasp (FxGrasp). dPCs that account for the highest amount of variance in the per-marginalization neural activity are shown here. These variances are included in brackets next to each component number. Vertical bars indicate the start and end of the go phase. Horizontal bars indicate time points at which the decoder axes of the pictured components classified forces (row 2), grasps (row 3), or force-grasp pairs (row 4) significantly above chance. B, Summary of variances accounted for by the top 20 dPCs and PCs from each session. Here, the variance accounted for by the dPCs approaches the variance accounted for by traditional PCs. Horizontal dashed lines indicate total signal variance, excluding noise. Row 2 shows the variance of each individual component, separated by marginalization. C, Go-phase activity within a two-dimensional PCA space. Estimated force axes within the low-dimensional PCA spaces are shown as blue lines. Here, c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch. D, Encoding model performances. The task-dependent components of neural feature activity were fit to the additive, scalar, and combined encoding models via cross-validated ordinary least squares regression. Tables contain the fit model coefficients for each session. Bar graphs indicate mean R2 values for each model over 100 iterations of Monte Carlo leave-group-out cross-validation. Error bars indicate SDs across iterations. Stars indicate statistically significant differences between model R2 values; **p < 0.01 and ***p < 0.001.
- Figure 6.
Time-dependent classification accuracies for force (rows 1–2) and grasp (row 3). Data traces were smoothed with a 100-ms boxcar filter to aid in visualization. Shaded areas surrounding each data trace indicate the SD across 240 session-runs for most trials in participant T8, 40 session-runs for elbow extension trials in participant T8, and 40 session-runs in participant T5. Gray shaded areas indicate the upper and lower bounds of chance performance over S × 100 shuffles of trial data, where S is the number of sessions per participant. Time points at which force or grasp is decoded above the upper bound of chance are deemed to contain significant force-related or grasp-related information. Blue shaded regions indicate the time points used to compute go-phase confusion matrices in Figure 7. Here, c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch. Time-dependent classification accuracies for individual force levels and grasp types are shown in Extended Data Figure 6-1. Grasp classification accuracies, separated by number of attempted grasp types, are presented in Extended Data Figure 6-2. Force classification accuracies, separated by individual session, are presented in Extended Data Figure 6-3.
- Figure 7.
Go-phase confusion matrices. A, Time-dependent classification accuracies (shown in Fig. 6) were averaged over go-phase time windows that commenced when performance exceeded 90% of maximum and ended with the end of the go phase. These yielded mean trial accuracies, which were then averaged over all session-runs in each participant. Overall force and grasp classification accuracies are indicated above each confusion matrix. SDs across multiple session-runs are indicated next to mean accuracies (cp = closed pinch, op = open pinch, rp = ring pinch, pow = power, elb = elbow extension). Statistical comparisons between the achieved classification accuracies are shown in Extended Data Figure 7-1. B, Confusion matrices, now separated by the grasps (c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch, power, elbow) that participants T8 (row 1) and T5 (row 2) used to attempt producing forces. Statistical comparisons between the achieved force accuracies are shown in Extended Data Figures 7-2, 7-3.
- Figure 8.
Go-phase force classification accuracy for novel (test) grasps. Within each session (rows), dPCA force decoders were trained on neural data generated during all grasps, excluding a single leave-out grasp type (columns). The force decoder was then evaluated over the set of training grasps (gray bars), as well as the novel leave-out grasp type (white bars). Stars indicate statistically significant differences in performance between training and novel grasps; **p < 0.01, ***p < 0.001. Error bars indicate the 95% confidence intervals. The horizontal dotted line in each panel indicates upper bound of the empirical chance distribution for force classification. Here, c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch.
Tables
Session Number Participant Post-Implant Day Number of Blocks per Grasp Closed Pinch Open Pinch Ring Pinch Power Elbow 1 T8 Day 735 11 — — 10 — 2 T8 Day 771 5 5 5 5 — 3 T8 Day 774 6 5 5 5 — 4 T8 Day 788 5 5 5 5 — 5 T8 Day 802 4 4 4 4 5 6 T8 Day 956 4 4 4 4 — 7 T5 Day 390 4 4 4 4 — Session information for participants T8 and T5, including the number of blocks per grasp type.
Extended Data Figure 2-1
Exemplary TC and SBP features tuned to task parameters of interest in participant T5, presented as in Figure 2. Note the presence of sharp activity peaks during the prep and stop phases of the trial, which were due to the presence of visual cues (Rastogi et al, 2020). Download Figure 2-1, TIF file.
Extended Data Figure 5-1
Neural population-level activity patterns for all sessions, presented as in Figure 5A–C. A, dPCs isolated from all individual sessions of neural data. B, Summary of variances accounted for by the top 20 dPCs from each exemplary session. Pie charts indicate the percentage of total signal variance accounted for by each marginalization. Total signal variance was computed with (left) and without (right) the condition-independent portion of the signal, as a basis of comparison to Figure 4 of the main text. C, Go-phase activity within two-dimensional PCA space. This figure shows dPCs, variances, and PCA plots for all recorded sessions. Corresponding encoding model performances for all recorded sessions appear in Extended Data Figure 5-2. Download Figure 5-1, TIF file.
Extended Data Figure 5-2
Encoding model performances, presented as in Figure 5D. Download Figure 5-2, TIF file.
Extended Data Figure 6-1
Time-dependent classification accuracies for individual force levels and grasp types. A, Time-dependent classification accuracies for force (row 1) and grasp (row 2), separated by force class and grasp class, respectively. Data traces were smoothed with a 100-ms boxcar filter to aid in in visualization. Shaded areas surrounding each data trace indicate the SD across 240 session-runs during most trials in participant T8, 40 session-runs during elbow extension trials in participant T8, and 40-session runs in participant T5. Gray shaded regions indicate the upper and lower bounds of chance performance over S × 100 shuffles of trial data, where S is the number of sessions per participant. Blue shaded regions indicate the time points used to compute go-phase confusion matrices. B, Time-dependent force classification accuracies during individual grasps in participants T8 (row 1) and T5 (row 2). Blue shaded regions indicate the time points used to compute go-phase confusion matrices. Decoding performances were averaged over S × 40 session runs, where S is the number of sessions per participant. Download Figure 6-1, TIF file.
Extended Data Figure 6-2
Time-dependent grasp classification accuracies by number of grasps attempted per session in participant T8. Data traces were smoothed with a 100-ms boxcar filter to aid in in visualization. Shaded areas surrounding each data trace indicate the SD across 40 runs during each session in participant T8. Gray shaded regions indicate the upper and lower bounds of chance performance over 100 shuffles of trial data per session. Intended grasp is classified above chance performance at all trial time points, regardless of the number of grasps to be decoded. Download Figure 6-2, TIF file.
Extended Data Figure 6-3
Time-dependent force classification accuracies by force level, per session, in participant T8. Data traces were smoothed with a 100-ms boxcar filter to aid in in visualization. Shaded areas surrounding each data trace indicate the SD across 40 runs during each session in participant T8. Gray shaded regions indicate the upper and lower bounds of chance performance over 100 shuffles of trial data per session. Intended grasp is classified above chance performance at all trial time points, regardless of the number of grasps to be decoded. Download Figure 6-3, TIF file.
Extended Data Figure 7-1
Statistics for go-phase force and grasp classifications accuracies. A, Force classification accuracy histograms (row 1) and corrected p values (row 2). Hard and light forces are classified significantly more accurately than medium forces across all sessions (p < 0.05). B, Grasp classification accuracy histograms (row 1) and corrected p values (row 2). Decoding performance differed significantly between grasps across all sessions. Download Figure 7-1, TIF file.
Extended Data Figure 7-2
Statistics for go-phase force classification accuracies within individual grasp types. A one-way ANOVA was implemented on force classification accuracies achieved during different grasp types. A, Force classification accuracy histograms. B, p values between force pairs, corrected for multiple comparisons across grasps and sessions using the Benjamini–Hochberg procedure. Within each grasp, hard and light forces were classified more accurately than medium forces across all sessions (p < 0.05). Download Figure 7-2, TIF file.
Extended Data Figure 7-3
Statistics for go-phase force classification accuracies within individual force levels. A one-way ANOVA was implemented on the force classification accuracies achieved during different grasp types. A, Force classification accuracy histograms, color-coded by the grasp type used to produce each force level. B, p values between pairs of grasps used to produce each individual force level, corrected for multiple comparisons across forces and sessions using the Benjamini–Hochberg procedure. The decoding performance for each discrete force level was significantly different across grasps (p < 0.05), indicating that grasp type affected force decoding performance. Download Figure 7-3, TIF file.
In this issue
