Article Figures & Data
Figures
- Figure 1.
General framework. In a stiffness discrimination task composed of trials (Farajian et al., 2023), we recorded the participants’ action signals and perceptual responses. We used the perceptual responses to compute metrics describing the participants’ perception. Additionally, we designed models that received each trial’s action signals as input and outputted the predicted perceptual response of that trial. In relevant models, the participants’ real perceptual responses were used as true labels for training. We used the predicted perceptual responses to compute the same metrics describing the participants’ predicted perception. Lastly, we compared the predicted metrics to the real ones to assess the performance of the models.
- Figure 2.
Experimental setup. a, The participants sat in front of a virtual reality system and used a haptic device to make downward probing movements into virtual objects. b, Side view of the skin stretch device, which was mounted on the end of the haptic device. The vertical movement of the tactors (red circles) created the artificial skin stretch. c, Side view of the skin stretch device grasped by the participant, who placed her thumb and index finger on the tactors. d, Back view of the skin stretch device, grasped by the participant. The grip force between the participant’s thumb and index finger was measured using a force sensor.
- Figure 3.
Signals for one trial. a, Raw and b, Filtered. The top graph shows the position, the second the velocity, and the bottom is the grip force. Gray indicates an interaction with the comparison object, and pink is the interaction with the standard object. The shaded gray area in the position signals indicates the portions of the signal in which participants interacted with the virtual objects (i.e., were in the negative half of the vertical axis). Only these portions of the signals were extracted and are shown using solid, darker lines in (b). The portions of the signal that were not used (participants were above the boundary of the object) are dashed in (b).
- Figure 4.
Synthetic psychometric curves. These are psychometric curves used for demonstration and explanation of the metrics, and are not the results of a specific participant. The pale-colored curves are for the force condition and the dark-colored curves are for the stretch condition. The abscissa is the difference between the stiffness levels of the two virtual objects, and the ordinate is the probability of choosing that the comparison object was stiffer. The blue curves show the hypothetical results of one participant, where the rightward shift of the dark blue curve indicates an increased stiffness perception due to the skin stretch. The red curves are the predicted results of the participant.
- Figure 5.
Artificial neural network. The comparison and standard signals were each inputted into identical neural networks. The network outputs were subtracted and inputted into a Dense layer with one neuron and a sigmoid activation. This layer classified the trial as 0 if the predicted response was comparison, and as 1 if standard was the predicted response.
- Figure 6.
Neural network results. a–d, Real and predicted psychometric curves of four participants. The blue curves were created using the participants’ real responses, and the red curves were created using the network’s predictions. The abscissa is the difference between the stiffness levels of the two virtual objects, and the ordinate is the probability of choosing that the comparison object was stiffer. The pale curves represent the force condition, and the dark curves are the stretch condition. a–b, Participants predicted well by the network. c–d, Participants whose perceptual effects were not predicted well. e, The real (blue shades) and predicted (red shades) average PSE of all 38 participants for the force and stretch conditions. The bars show the average effects, and the black error bars are the 95% confidence intervals. f, The results of all 38 participants. The gray stars show each participant’s predicted ΔPSE relative to their real one. The black line shows the regression of the predicted ΔPSEs against the real ΔPSEs. The dashed black lines show zero on the abscissa and ordinate. The gray dashed line is the ideal model, that is, a model with an intercept of zero and a slope of one.
- Figure 7.
Errors and Certainty. a-c, Network errors relative to the participants’ perceptual responses as a function of the comparison stiffness level for one of the five runs. The percentage of errors for each stiffness level was computed out of the total number of trials with that stiffness level in the given condition. a, All the trials together. b, The force trials alone. c, The artificial skin stretch trials alone. d–f, The average outputs of the network as a function of the comparison stiffness level for one of the five runs. This reflects the certainty, or loss value, of the network for each comparison level. The average outputs for correct predictions are indicated with black stars, and for mistaken predictions, with gray squares. In trials with the response standard, outputs closer to 1 reflect better performance, whereas for trials in which the response was comparison, outputs closer to 0 are superior. d, All the trials together. e, The force trials alone. f, The artificial skin stretch trials alone.
- Figure 8.
Maximum penetration model results. a, Real and predicted psychometric curves of one of the participants. The blue curves are those created using the participant’s real responses, whereas the red curves were created using the predictions of the Maximum Penetration Model. In these graphs, the abscissa is the difference between the stiffness levels of the two virtual objects, and the ordinate is the probability of choosing that the comparison object was stiffer. The pale curves represent the force condition, and the dark curves are the stretch condition. A rightward shift of the dark curve indicates an increase in stiffness perception due to the skin stretch, whereas a leftward shift indicates a decrease. b, The real (blue shades) and predicted (red shades) average PSE of all 38 participants for the force and stretch conditions. The bars show the average effects, and the black error bars are the 95% confidence intervals. c, The results of all 38 participants. The gray stars show each participant’s predicted ΔPSE relative to their real one. The black line shows the regression of the predicted ΔPSEs against the real ΔPSEs. The dashed black lines show the zero on the abscissa and ordinate. The gray dashed line shows the ideal model, that is, a model with an intercept of zero and a slope of one.
- Figure 9.
Contribution of action signals to the prediction. a, No grip force. b, Only grip force. c, Each motion signal removed in turn. Pale blue is no position; the medium blue is no velocity; and the dark blue is no acceleration. d, Position and grip force (i.e., no velocity and acceleration). Each graph shows the results for one of the five runs, the average and standard deviation of all five runs are reported in Table 2. The dashed black lines show the zero on the abscissa and ordinate. The gray dashed line shows the ideal model, that is, a model with an intercept of zero and a slope of one. In a, b, and d, the gray stars show each participant’s predicted ΔPSE relative to their real one.
- Figure 10.
Positive and negative stretch. a–c, Examples of real and predicted psychometric curves. The abscissa is the difference between the stiffness levels of the two virtual objects, and the ordinate is the probability of choosing that comparison was stiffer. The blue and purple curves were created using the participants’ real responses, whereas the red and orange curves were created from the network’s predictions. The blue and red represent the positive stretch session (P), and purple and orange are the negative stretch session (N). The pale curves represent the force conditions (F), and the dark curves are the stretch conditions (S). d, The average real and predicted PSE for each condition for one of the five runs, where the colors correspond to those of the psychometric curves. The error bars show the 95% confidence intervals. e, The predicted ΔPSE as a function of the real ΔPSE for one of the five runs. Each symbol represents a participant, where the blue are the positive stretch session, and the purple are the negative stretch session. The regression line is shown in black. The dashed black lines show the zero on the abscissa and ordinate. The gray dashed line shows the ideal model – a model with an intercept of zero and a slope of one.
Tables
Regression (Predicted ΔPSE Against Real ΔPSE) PSE Error [N/m] JND [N/m] Intercept [N/m] Slope [au] p- value Force (Real = 0.16) Stretch (Real = 20.64) Force (Real = 12.13) Stretch (Real = 14.95) Model Accuracy Predicted Error Predicted Error Maximum penetration 69.51 −2.67 −0.15 0.13 −1.84 24.35 16.97 −4.84 18.93 −3.97 Maximum velocity 54.15 11.5 −0.06 0.96 −0.51 9.67 73.20 −61.07 55.26 −40.32 Average velocity 50.19 −16.95 −3.46 0.58 −79.10 29.16 2447.30 −2435.13 −133.31 148.25 Maximum grip force 59.18 −5.75 −0.18 0.86 11.41 41.33 136.58 −124.45 94.65 −79.70 Logistic regression 64.54 ± 0.03 6.85 −0.30 0.77 −70.57 −50.81 48.78 −36.66 61.23 −46.28 Regression (Predicted ΔPSE Against Real ΔPSE) PSE Error [N/m] JND [N/m] Intercept [N/m] Slope [au] p-value Force (Real = 0.16) Stretch (Real = 20.64) Force (Real = 12.13) Stretch (Real = 14.95) Training Signals Accuracy Predicted Error Predicted Error All four signals 77.91 ± 0.003 12.53 ± 1.02 0.30 ± 0.03 0.06 ± 0.01 0.50 ± 0.27 2.17 ± 1.31 9.54 ± 0.35 2.59 ± 0.35 14.11 ± 0.36 0.84 ± 0.36 No grip force 75.54 ± 0.002 0.92 ± 0.77 −0.13 ± 0.03 0.10 ± 0.09 −4.07 ± 0.62 18.13 ± 0.56 10.71 ± 0.25 1.41 ± 0.25 10.34 ± 0.39 4.61 ± 0.39 Only grip force 68.48 ± 0.001 16.89 ± 1.97 1.11 ± 0.15 0.02 ± 0.04 −13.10 ± 2.34 −32.33 ± 2.28 48.57 ± 6.51 −36.44 ± 6.51 41.04 ± 1.86 −26.09 ± 1.86 No position 76.14 ± 0.001 12.27 ± 0.86 0.33 ± 0.03 0.03 ± 0.02 0.97 ± 1.20 2.38 ± 1.27 13.54 ± 0.38 −1.41 ± 0.38 15.57 ± 0.20 −0.62 ± 0.20 No velocity 77.89 ± 0.002 11.34 ± 0.46 0.35 ± 0.01 0.04 ± 0.01 0.04 ± 0.42 2.02 ± 0.40 9.55 ± 0.13 2.58 ± 0.13 14.35 ± 0.24 0.60 ± 0.24 No acceleration 76.67 ± 0.001 12.56 ± 0.80 0.43 ± 0.03 0.02 ± 0.01 0.60 ± 0.42 −0.32 ± 0.37 11.51 ± 0.13 0.62 ± 0.13 16.32 ± 0.22 −1.37 ± 0.22 Grip force and position 76.25 ± 0.003 13.55 ± 0.57 0.48 ± 0.03 0.02 ± 0.01 −0.02 ± 0.13 −2.90 ± 0.89 12.78 ± 0.42 −0.66 ± 0.42 17.55 ± 0.41 −2.61 ± 0.41 Bold is significant regression.
Regression (Predicted ΔPSE Against Real ΔPSE) PSE Error [N/m] JND Error [N/m] Accuracy Intercept [N/m] Slope [au] p-value Force Positive (Real = 0.16) Stretch Positive (Real = 20.64) Force negative (Real = 0.06) Stretch Negative (Real = 7.81) Force positive (Real = 12.13) Stretch Positive (Real = 14.95) Force negative (Real = 12.38) Stretch Negative (Real = 14.64) 75.58 ± 0.001 16.01 ± 0.51 0.31 ± 0.03 0.007 ± 0.007 2.11 ± 0.37 −1.73 ± 0.70 3.20 ± 0.18 −5.40 ± 0.34 −1.04 ± 0.14 −2.10 ± 0.27 −1.54 ± 0.22 −1.64 ± 0.33 Bold is significant regression.
In this issue
