Visual and Tactile Sensory Systems Share Common Features in Object Recognition

Similarity matrices. A, Average group similarity matrix for visual similarity judgment. B, Average group similarity matrix for tactile similarity judgment. The color codes for the similarity ratings corresponded to the numbers, ranging from 1 (dissimilar, dark blue) to 7 (identical, dark red). Numbers on the x- and y-axes refer to the digital embryos in each category (eight objects per category) according to Figure 1A.

Two-dimensional visual and tactile perceptual spaces. A, The stress values for both modalities were calculated for 1–10 dimensions. The elbow indicates that two data dimensions are sufficient to explain the visual and the tactile perceptual space. B, Two-dimensional visual perceptual space (Extended Data Fig. 3-1A,C, one- and three-dimensional visual perceptual spaces). C, Two-dimensional tactile perceptual space (Extended Data Fig. 3-1B,D, one- and three-dimensional tactile perceptual spaces). The numbers refer to the object numbers in each category according to Figure 1A. Contrast level codes for different categories; black, category 1; gray, category 2.

Euclidian distance. The average distance between pairs of objects within a category (black bars) is significantly smaller than the distance between pairs of objects from different categories (gray bars) for both modalities. Error bars represent the SEM. **p < 0.0001.

Questionnaires. At the end of each similarity judgment test, participants were asked to rate the importance of features to determine which features played the main role in their similarity judgments. In addition to the weight, color, material, global patterning, and texture, we listed further details describing the shape of digital embryos: branch size, branch pattern, number of branches, global shape, and curvature. The results for both modalities indicated that the shape features played a major role. Features such as weight, size, and the pattern of branch distributions were significantly more important for the tactile similarity judgment experiment than for the visual. Bars represent the mean ratings across all participants over the visual (gray) and tactile (black) modalities (0 means no importance; 6 means very important). Error bars represent the SEM. *p < 0.01.

Mean d values (Procrustes fit error). A, Fits between computational measures of a single feature with visual and tactile maps. B, Fits between computational measures of a combination of features with visual and tactile maps. F1–F17 indicate the number of single features. N1–N17 present a combination of features that are listed in Table 2. Asterisks demonstrate that a reconstructed map of a feature or combination of features is significantly different from perception in human behavior (p < 0.01). MD indicates cross-fitting error.