Article Figures & Data
- Figure 1.
A, Schematic of the avatar generation process. B, Overview of stimulus categories. Render types, columns left to right, Wireframe, grayscale, furless, naturalistic avatar, real monkey. Expressions, rows top to bottom, Neutral, fear grin, lip smacking, threat, artificial blowing. Images depict extreme frames of the expressions, which were also used for the static videos. C, Example outlines of ROIs, within which fixations were analyzed, with overlaid example scanpaths on a wireframe avatar, neutral face (top left), grayscale avatar, blowing face (bottom left), furless avatar, lip-smacking face (top right), naturalistic avatar, fear grinning face (middle right), and real, threatening face (bottom right). Blue, Entire face ROI. Orange, Eyes ROI. Yellow, Nose ROI. Purple, Mouth ROI. White, Scanpath. Black star, First fixation. Black diamonds, Subsequent fixations. ROIs were manually drawn using the MATLAB impoly function and the coordinates of the ROI polygon were subsequently extracted using the getPosition function. D, Heatmap of all fixations of all monkeys during the experiment. Heatmaps were created by plotting all fixations and convolving the image first in x, then in y direction with a Gaussian function of standard deviation 10 pixels, taking the duration of each fixation interval as the amplitude.
- Figure 2.
Number of fixations on different render types (N = 8). A, Boxplots show fixation numbers per render type (render type effect size: χ2(4) = 72.60, p < 0.001). B, Fixation numbers for all monkeys individually. Markers indicate within-subject mean ± SEM. C, Median fixation numbers separated by expression; effect was significant within each expression category except for blowing (marginally significant). Neutral: χ2(4) = 19.09, p < 0.001; fear: χ2(4) = 17.80, p = 0.0013; lip smack: χ2(4) = 19.39, p < 0.001; threat: χ2(4) = 19.90, p < 0.001; blowing: χ2(3) = 7.37, p = 0.061. D, Boxplots show fixation numbers on dynamic and static videos within each render type category. Significant differences between dynamic and static videos were found for grayscale avatars only. Wireframe: χ2(1) = 0.026, p = 0.87; grayscale: χ2(1) = 5.77, p = 0.016; furless: χ2(1) = 0.026, p = 0.87; avatar: χ2(1) = 0.40, p = 0.53; real: χ2(1) = 1.13, p = 0.29; *p < 0.05, **p < 0.01, ***p < 0.001.
- Figure 3.
Number of fixations on different expressions (N = 8). A, Boxplots show fixation numbers per dynamic expressions (left), expression effect size: χ2(3) = 19.27, p < 0.001; and static expressions (right), expression effect size: χ2(3) = 25.82, p < 0.001. B, Fixation numbers for all monkeys individually, dynamic expressions (left) and static expressions (right). Markers indicate within-subject mean ± SEM. C, Median fixation numbers separated by render type. Left, Dynamic expressions: wireframe: χ2(4) = 9.02, p = 0.061; gray: χ2(4) = 8.51, p = 0.075; no fur: χ2(4) = 7.16, p = 0.13; avatar: χ2(4) = 13.01, p = 0.56; real: χ2(3) = 2.54, p = 0.47. Right, Static expressions, effect was significant within the no fur category and marginally significant within the gray category: wireframe: χ2(4) = 6.80, p = 0.15; gray: χ2(4) = 9.48, p = 0.050; no fur: χ2(4) = 11.56, p = 0.021; avatar: χ2(4) = 4.38, p = 0.36; real: χ2(3) = 2.88, p = 0.41; *p < 0.05, **p < 0.01, ***p < 0.001.
- Figure 4.
Monkeys’ average fixation numbers on each side during the preferential looking paradigm (experiment 2), revealing strong side biases of all monkeys except monkey P.
- Figure 5.
HRV, measured as RMSSD (N = 5). A, All dynamic expressions compared, RMSSD tended to be lower in the threatening condition, indicating elevated arousal. B, Dynamic threatening versus static threatening expressions. C, Grayscale dynamic expressions only; *p < 0.05, **p < 0.01, ***p < 0.001.
- Figure 6.
Average probability to show lip smacking, fear grinning, agitation (teeth grinding/tension yawning), or no reaction when viewing each render type. Monkeys (N = 3) were most likely to lip smack when seeing the real monkey and the naturalistic avatar (wireframe < real, p < 0.001; furless < real, p = 0.0018; wireframe < avatar, p = 0.052). The probability for no reaction was highest for the unrealistic wireframe head (wireframe > gray, p = 0.024; wireframe > furless, p = 0.011; wireframe > avatar, p < 0.001; wireframe > real, p < 0.001; gray > real, p = 0.017; furless > real, p = 0.035). The highest probability to fear grin occurred toward the furless avatar (wireframe < furless, p < 0.001, wireframe < avatar, p = 0.041; real < furless p = 0.041). Differences in agitated reactions were not significant.
- Table 1
Render type effect on all looking parameters investigated: cumulative fixation duration (FixDur), fixation number (FixNum), mean fixation duration (MeanFix), and total looking time (LookT) on the entire face, the eyes and the mouth, as well as exploration distance (ExplorDistance) and the feature index
Lookingparameter Wireframe Grayscale Furless Avatar Real χ2 (df = 4),n = 64, p value median (IQR) Diff median (IQR) Diff median (IQR) Diff median (IQR) Diff median (IQR) Diff FixDur face 1372.38
(1004.95–1528.14)– 1264.38
(909.91–1491.12)R 1307.35
(972.22–1500.50)– 1287.79
(1067.57–1497.17)– 1291.11
(1107.00–1503.56)G 14.34, 0.0063 FixNum face 5.25 (4.10–5.88) G,F 4.50 (3.47–5.15) W,A,R 4.65 (3.87–5.30) W,A,R 5.05 (4.29–5.54) G,F,R 5.24 (4.59–5.81) G,F,A 72.60, 0.000 MeanFix face 257.90
(202.76–297.55)G,F 281.20
(219.47–335.81)W,R 279.79
(215.82–318.47)W 266.05
(223.89–317.37)– 255.79
(209.76–304.01)G 14.31, 0.000 LookT face 1747.15
(1481.48–1904.80)– 1645.17
(1332.39–1805.54)R 1632.71
(1325.82–1818.56)R 1659.63
(1464.91–1869.56)– 1722.50
(1490.59–1859.79)G,F 20.16, 0.000 FixDur eyes 131.69
(43.25–333.02)– 154.24
(40.20–340.94)R 208.69
(79.64–424.49)R 148.73
(54.76–258.94)– 120.68
(57.80–196.25)G,F 22.76, 0.000 FixNum eyes 0.69 (0.23–1.45) – 0.79 (0.14–1.60) – 0.88 (0.41–1.63) R 0.73 (0.19 –1.16) – 0.59 (0.30–0.95) F 13.96, 0.0074 MeanFix eyes 87.69
(34.21–160.47)F 99.03
(27.18–185.81)– 111.18
(41.15–200.42)W,R 86.06
(46.71–137.05)– 88.52
(41.03–113.22)F 17.82, 0.0013 LookT eyes 206.27
(68.00–450.86)– 233.94
(70.78–433.51)R 264.91
(113.93–543.33)R 225.89
(88.39–331.98)– 150.89
(85.44–235.24)G,F 22.95, 0.000 FixDur mouth 114.21
(44.21–348.00)R 85.50
(22.00–238.72)R 83.55
(36.43–255.68)R 134.61
(37.64–322.05)R 295.09
(154.16–502.11)W,G,F,A 31.41, 0.000 FixNum mouth 0.47 (0.20–1.10) R 0.30 (0.13–0.76) R 0.35 (0.15–0.88) R 0.60 (0.18–1.00) R 1.05 (0.65–1.75) W,G,F,A 49.13, 0.000 MeanFix mouth 88.65
(38.25–170.35)R 66.22
(21.51–134.25)R 72.56
(21.61–167.42)R 85.35
(34.71–180.25)R 184.02
(121.85–234.05)W,G,F,A 32.15, 0.000 LookT mouth 185.02
(89.82–508.29)R 140.38
(48.50–486.70)R 190.06
(70.80–458.94)R 216.90
(76.28–545.16)R 663.86
(357.08–890.90)W,G,F,A 79.75, 0.000 ExplorDistance 10.38 (6.17–13.22) F 10.57 (5.25–12.73) A,R 10.91 (6.98–12.67) W,A,R 12.57 (7.26–14.25) G,F 11.82 (7.52–14.12) G,F 50.91, 0.000 Feature index –0.13 (–0.45–0.064) – –0.23 (–0.48–0.19) – –0.27 (–0.42–0.15) – –0.29 (–0.47–0.070) – –0.32 (–0.49– 0.00) – 7.16, 0.13 Table shows group medians and interquartile range (IQR) of all eight subjects per condition, results of Friedman’s ANOVA (last column) with test statistic value (χ2), degrees of freedom (df), number of values per condition included in the statistical analysis (n), and significance level (p value) and shows from which groups the respective group differed significantly (Diff, p < 0.05) according to Dunn and Sidák’s post hoc multiple comparisons approach (W = wireframe, G = grayscale, F = furless, A = avatar, R = real). To be able to apply Friedman’s ANOVA, trials with a blowing avatar were excluded from the analysis to assure the same number of expressions per render type category (no blowing expression in the real video). Effects were regarded as statistically significant at p < 0.05; p values <0.001 were rounded to 0.000.
- Table 2
Expression effect for all data and for dynamic and static conditions separately on all looking parameters investigated: Cumulative fixation duration (FixDur), fixation number (FixNum), mean fixation duration (MeanFix), total looking time (LookT) on the entire face, the eyes and the mouth, as well as exploration distance (ExplorDistance) and the feature index
Lookingparameter Neutral Fear Lip smack Threat Blowing χ2 (df = 3), n = 80 (all)/n = 40 (dyn./ stat.), p value median (IQR) Diff median (IQR) Diff median (IQR) Diff median (IQR) Diff median (IQR) Diff FixDur face 1248.40
(945.96–1504.73)T 1285.90
(1007.32–1509.05)- 1274.78
(1036.78–1472.54)- 1359.36
(1116.01–1521.50)N 1295.06
(906.30–1512.56)- 13.19, 0.0043 Dynamic 1252.29
(1024.79–1499.55)T 1261.65
(1023.37–1481.34)T 1334.24
(1089.78–1495.33)- 1413.88
(1187.31–1557.18)N,F 1368.56
(866.48–1495.15)- 11.61, 0.0088 Static 1231.20
(895.70–1512.90)- 1359.37
(1007.33–1515.76)- 1219.72
(964.70–1470.73)- 1300.02
(1045.58–1474.63)- 1248.46
(946.79–1531.31)- 9.81, 0.020 FixNum face 4.60 (3.74–5.34) F,T 4.78 (4.17–5.59) N,T 4.75 (4.00–5.50) T 5.16 (4.39–5.75) N,F,L,B 4.75 (3.79–5.33) T 36.13, 0.000 Dynamic 4.70 (3.54–5.29) T 4.67 (4.10–5.35) T 5.00 (4.06–5.64) - 5.11 (4.20–5.65) N,F 4.79 (3.61–5.32) - 19.27, 0.000 Static 4.59 (3.95–5.40) F,T 5.17 (4.21–5.66) N,L 4.68 (3.93–5.29) F,T 5.29 (4.58–5.88) N,L 4.75 (3.79–5.38) - 25.82, 0.000 MeanFix face 272.72
(215.33–327.00)- 268.26
(221.70–314.29)- 271.51
(214.84–326.07)- 262.75
(214.94–314.11)- 272.27
(228.53–329.26)- 1.67, 0.64 Dynamic 276.33
(222.06–359.12)- 270.66
(224.89–318.06)- 262.50
(217.53–316.61)- 277.96
(228.70–337.22)- 275.36
(230.03–337.23)- 3.57, 0.31 Static 264.10
(201.13–321.86)- 266.48
(203.82–301.40)- 273.47
(208.47–342.47)- 250.87
(205.81–290.07- 263.72
(218.54–312.33)- 8.04, 0.045 LookT face 1583.35
(1324.88–1822.86)F,T 1676.00
(1482.22–1859.79)N,B 1651.82
(1414.69–1816.48)T 1741.56
(1515.20–1897.24)N,L,B 1659.39
(1315.08–1804.15)F,T 21.98, 0.000 Dynamic 1583.35
(1356.10–1824.22)T 1655.18
(1487.83–1847.60)T 1709.45
(1490.44–1860.99)- 1769.91
(1559.55–1913.73)N,F,B 1691.35
(1310.83–1796.10)T 17.55, 0.000 Static 1579.56
(1224.25–1814.67)F 1723.87
(1473.90–1882.95)N,L,B 1605.48
(1343.86–1791.33)F 1720.75
(1484.35–1849.24)- 1614.57
(1355.80–1841.81)F 12.99, 0.0047 FixDur eyes 132.45
(47.45–309.08)- 137.44
(52.25–295.85)- 148.83
(58.52–277.27)- 157.44
(49.95–295.45)- 161.70
(36.61–337.46)- 0.61, 0.89 Dynamic 132.45
(40.79–269.58)- 171.50
(52.25–267.40)- 141.52
(58.52–254.95)- 163.75
(64.73–280.55)- 86.83
(20.15–282.23)- 4.92, 0.18 Static 146.00
(53.75–316.96)- 108.55
(49.13–315.10)- 174.06
(58.98–355.03)- 152.92
(43.42–297.25)- 186.05
(85.84–346.96)- 4.24, 0.24 FixNum eyes 0.68 (0.21–1.25) - 0.61 (0.20–1.27) - 0.75 (0.32–1.38) - 0.75 (0.27–1.41) - 0.65 (0.15–1.40) - 0.16, 0.98 Dynamic 0.60 (0.14–1.13) - 0.72 (0.31–1.24) - 0.65 (0.33–1.14) - 0.71 (0.34–1.40) - 0.44 (0.11–1.25) - 3.21, 0.36 Static 0.80 (0.30–1.38) - 0.56 (0.17–1.31) - 0.76 (0.30–1.50) - 0.84 (0.24–1.46) - 0.79 (0.24–1.56) - 4.48, 0.21 MeanFix eyes 84.13
(37.22–155.71)- 91.27
(42.00–158.89)- 92.07
(36.78–166.40)- 104.53
(37.88–153.84)- 97.34
(33.61–173.51)- 0.090, 0.99 Dynamic 78.45
(37.22–149.58)- 93.22
(44.70–152.67)- 89.26
(31.24–161.60)- 108.00
(34.13–170.79)- 54.27
(20.15–158.55)- 5.93, 0.12 Static 97.98
(37.33–159.86)- 74.61
(38.64–158.89)- 99.60
(49.19–182.44)- 85.94
(42.83–136.75)- 100.56
(46.76–176.23)- 6.73, 0.081 LookT eyes 207.33
(78.79–394.69)- 190.17
(79.54–391.33)- 207.94
(85.69–406.61)- 228.67
(73.89–359.25)- 200.88
(81.15–446.29)- 1.18, 0.76 Dynamic 198.74
(72.49–351.97)- 204.37
(79.54–371.99)- 190.75
(75.01–342.14)- 236.72
(86.73–359.25)- 162.28
(42.97–412.26)- 2.31, 0.51 Static 229.50
(93.14–441.49)- 174.60
(85.16–423.28)- 221.06
(115.55–483.18)- 218.98
(73.89–359.33)- 236.95
(108.33–456.71)- 7.39, 0.060 FixDur mouth 42.77
(0.00–105.28)F,L,T 332.88
(164.00–492.01)N,L,B 97.60
(32.34–267.93)N,F,T 194.50
(86.36–435.15)N,L,B 40.36
(0.00–123.09)F,T 138.48, 0.000 Dynamic 53.08
(0.00–101.61)F,L,T 318.10
(175.22–425.38)N,L,B 169.83
(37.80–308.61)N,F,T 215.08
(63.03–431.31)N,L,B 46.55
(0.00–164.79)F,T 60.61, 0.000 Static 35.21
(0.00–105.60)F,T 364.00
(162.89–516.43)N,L,B 68.04
(19.36–147.10)F,T 191.39
(105.68–444.44)N,L,B 32.61
(0.00–75.89)F,T 82.44, 0.000 FixNum mouth 0.18 (0.00–0.35) F,L,T 1.10 (0.71–1.45) N,L,B 0.33 (0.13–0.80) N,F,T 0.76 (0.38–1.61) N,L,B 0.13 (0.00–0.44) F,T 154.06, 0.000 Dynamic 0.18 (0.00–0.30) F,L,T 1.15 (0.68–1.40) N,L,B 0.52 (0.19–0.86) N,F,T 0.73 (0.33–1.50) N,L,B 0.13 (0.00–0.53) F,T 68.47, 0.000 Static 0.17 (0.00–0.42) F,T 1.05 (0.75–1.69) N,L,B 0.25 (0.13–0.69) F,T 0.83 (0.50–1.69) N,L,B 0.13 (0.00–0.28) F T 91.15, 0.000 MeanFix mouth 32.29
(0.00–76.13)F,L,T 185.52
(99.68–250.33)N,L,B 81.24
(22.28–147.90)N,F,T 124.94
(68.72–212.58)N,L,B 37.44
(0.00–91.23)F,T 110.95, 0.000 Dynamic 31.64
(0.00–83.43)F,L,T 185.96
(115.80–260.35)N,L,B 108.88
(37.80–193.73)N,F 141.08
(54.68–231.49)N,B 46.55
(0.00–109.25)F,T 52.05, 0.000 Static 32.71
(0.00–76.13)F,T 183.11
(82.40–242.64)N,L,B 44.37
(19.36–106.97)F,T 105.57
(78.54–198.50)N,L,B 29.39
(0.00–66.74)F,T 61.35, 0.000 LookT mouth 95.07
(31.49–217.05)F,L,T 669.77
(389.74–839.84)N,L,T,B 167.30
(59.77–465.01)N,F,T 296.59
(144.95–679.65)N,F,L,B 75.64
(24.02–211.48)F,T 171.80, 0.000 Dynamic 98.30
(34.73–268.66)F,L,T 646.07
(408.83–790.35)N,L,T,B 203.35
(88.94–546.95)N,F 310.63
(130.80–723.90)N,F,B 103.41
(33.61–228.45)F,T 81.12, 0.000 Static 93.84
(26.37–213.30)F,T 685.58
(375.59–910.24)N,L,B 131.00
(50.00–308.78)N,F,T 296.59
(162.84–672.35)N,F,L,B 59.84
(20.94–160.83)F,T 94.59, 0.000 ExplorDistance 10.34
(5.76–12.35)T 11.29
(7.24–13.24)T 10.57
(6.39–12.96)T 12.79
(7.97–15.43)N,F,L,B 10.51
(5.83–12.96)T 46.40, 0.000 Dynamic 9.65 (5.76–11.95) T 11.11 (7.24–13.53) T 11.36 (6.32–13.45) T 12.35 (6.97–15.56) N,F,L,B 10.40 (4.94–12.88) T 18.45, 0.000 Static 11.06 (5.82–12.41) T 11.29 (7.31–13.08) T 10.33 (6.39–12.05) T 13.09 (8.76–15.09) N,F,L,B 10.51 (7.48–13.28) T 29.19, 0.000 Feature index –0.40 (–0.66–0.049) F,L,T –0.14 (–0.40–0.12) N,L,B –0.32 (–0.50–0.061) N,F,T –0.12 (–0.40–0.20) N,L,B –0.39 (–0.57– 0.0058) F,T 56.15, 0.000 Dynamic –0.39 (–0.62–0.020) F,T –0.16 (–0.38–0.086) N,B –0.33 (–0.49–0.087) - –0.12 (–0.43–0.20) N,B –0.45 (–0.63– –0.055) F,T 23.85, 0.000 Static –0.40 (–0.72–0.077) F,T –0.10 (–0.42–0.20) N,L,B –0.30 (–0.54–0.054) F,T –0.10 (–0.36–0.20) N,L,B –0.35 (–0.54–0.0034) F,T 33.69, 0.000 Table shows group medians and interquartile range (IQR) of all eight subjects per condition, results of Friedman’s ANOVA (last column) with test statistic value (χ2), degrees of freedom (df), number of values per condition included in the statistical analysis (n), and significance level (p value) and shows from which groups the respective group differed significantly (Diff, p < 0.05) according to Dunn and Sidák’s post hoc multiple comparisons procedure (N = neutral, F = fear, L = lip smack, T = threat, B = blowing). Friedman’s ANOVAs did not include the blowing expressions to assure the same number of render types per expression category (no real blowing video). A second Friedman’s ANOVA was conducted leaving out the real videos (data not shown) and pairwise differences with the blowing expression are reported from this analysis. Effects were regarded as statistically significant at p < 0.05; p values <0.001 were rounded to 0.000.
- Table 3
Video type effect on all looking parameters investigated: cumulative fixation duration (FixDur), fixation number (FixNum), mean fixation duration (MeanFix), total looking time (LookT) on the entire face, the eyes and the mouth, as well as exploration distance (ExplorDistance) and the feature index
Lookingparameter Dynamic Static χ2 (df = 1), n = 192,
p valuemedian (IQR) median (IQR) FixDur face 1318.51 (1048.08–1512.81) 1269.04 (978.38–1497.71) 3.00, 0.083 FixNum face 4.87 (4.00–5.50) 4.82 (4.05–5.57) 0.89, 0.34 MeanFix face 273.05 (225.98–327.38) 263.35 (207.90–305.57) 21.33, 0.000 LookT face 1687.10 (1464.07–1862.00) 1655.48 (1400.45–1845.85) 2.08, 0.15 FixDur eyes 145.74 (42.89–265.92) 156.75 (57.86–326.73) 7.40, 0.0065 FixNum eyes 0.63 (0.21–1.25) 0.75 (0.22–1.47) 4.26, 0.039 MeanFix eyes 90.99 (32.88–157.91) 97.40 (43.25–163.75) 1.22, 0.27 LookT eyes 199.50 (72.57–363.35) 225.29 (94.66–443.80) 8.33, 0.0039 FixDur mouth 148.45 (37.80–322.05) 105.68 (32.49–306.04) 0.05, 0.82 FixNum mouth 0.43 (0.14–1.09) 0.41 (0.13–1.00) 0.60, 0.44 MeanFix mouth 99.79 (28.72–193.26) 75.88 (25.31–178.24) 6.08, 0.014 LookT mouth 216.93 (79.43–573.30) 204.05 (63.28–557.71) 2.08, 0.15 ExplorDistance 10.73 (6.23–13.56) 11.03 (7.07–13.49) 0.083, 0.77 Feature index –0.26 (–0.50–0.063) –0.23 (–0.49–0.097) 0.76, 0.38 Table shows group medians and interquartile range (IQR) of all eight subjects per condition and results of Friedman’s ANOVA (last column) with test statistic value (χ2), degrees of freedom (df), number of values per condition included in the statistical analysis (n), and significance level (p value). Groups were regarded as statistically significant at p < 0.05; p values <0.001 were rounded to 0.000.
In this issue
