A General Framework for Inferring Bayesian Ideal Observer Models from Psychophysical Data

The canonical Bayesian computation as in Figure 1 but expanded to a set of likelihood functions. The prior (A) is multiplied by the likelihood defined by a given measurement (B, shown for m₁ and m₂) to obtain the posterior (C). Note that the shape of the posteriors change for different likelihoods since the prior is non-Gaussian, but the posteriors are overall drawn to the largest probability region of the prior. In each panel, the heat map values represent probability with higher intensity mapping to higher probability. Identity lines are indicated with dashed black lines. Toolkit script: Fig2_2DBayesianDemo.m.

The distribution of sensory estimates arises from the variability in the measurement values about the expected value across trials (EV; i.e., the true stimulus value). A, On a given trial, a likelihood is defined around the observed measurement. Here, we plot the expected value of this likelihood for a given true stimulus, as well as other possible likelihoods that occur on a set of trials. The prior is shown for reference. The upward arrow indicates the true stimulus that used to generate the likelihood. B, The resulting posteriors for each trial are shown, along with downward arrows indicating the estimates ({x^} ) derived from these posteriors. C, Over many trials, these estimates (now indicated as upward arrows) create an estimate distribution, which can be predicted for a Bayesian ideal observer with a given prior and amount of sensory noise. When the prior is Gaussian, there is a closed form expression for this distribution. Toolkit script: Fig3_EstimateDistribution.m.

Graphical illustration of computing the observer’s psychometric curve for a 2AFC task. A, Computing a single point on the psychometric curve when x1=x2=3 for measurement noise variances σ12=0.75,σ22=0.5 and a prior with v = 0 and γ = 1.5. Dashed line (top) shows the measurement distribution p(m1|x1) and solid line (right) shows measurements distribution p(m|x₂). The 2D grayscale image shows the joint distribution of observer measurements given the stimuli x₁ and x₂, formed by the product of the two measurement distributions along the top and right. The white diagonal line is the observer’s decision boundary, corresponding to measurement values for which the inferred speeds are equal. The probability that the observer reports “yes” (i.e., that x₂ exceeded x₁) is the area above the decision boundary (point “A” in panel D). B, Same as panel A but with equal noise variances σm12=σm22=0.64 . C, Same as panel A but with noise variances σm12=0.5,σm22=0.75 . D, Full psychometric curves for the noise variances used in panels A–C, showing the probability that the observer reports “yes” as a function of the stimulus x₂. The points labeled A, B, C represent the sum of the probability above the diagonal in panels A–C.

Prior and posterior defined by a mixture of Gaussian components. A, The prior of a Bayesian observer (dark red line) can be modeled as a mixture of Gaussian components (light red lines). B, When combined with a Gaussian likelihood, the resulting posterior is also a mixture of Gaussians. Similar to the posterior resulting from a single Gaussian prior, the mixture of Gaussians posterior is biased relative to the likelihood. Likelihoods are shaded here for visual clarity. Toolkit script: Fig5_MoGprior.m.

An extension of Figure 4 to a long-tailed prior defined by a mixture of Gaussians (γ1=2,γ2=0.6 ν1=ν2=0,w1=w2=0.5 ), similar in appearance to the prior in Figure 7A. Here, the decision boundary representing T(m1,m2) is nonlinear because the different components of the prior have different levels of influence on the percept as m varies. A, As in Figure 4, the 2D grayscale image shows the joint distribution of the observer measurements given the stimuli x₁ and x₂, formed by the product of the two measurement distributions along the top and right. The white line is the observer's decision boundary. Here, x₁ = x₂ = 3 for measurement noise variances σ²₁ = 0.75, σ²₁ = 0.5. B, Same as panel A, but with equal noise variances σ²_m1 = σ²_m2 = 0.64. C, Same as panel A, but with measurement noise variances σ²₁ = 0.75, σ²₁ = 0.5. Toolkit script: Fig6_MoGGauss_graphicalDemo.m. D, Full psychometric curves for the noise variances used in panels A–C, showing the probability that the observer reports “yes” as a function of the stimulus X₂. The points labeled A, B, C represent the sum of the probability above the diagonal in panels A–C.

Two example methods for reducing the number of parameters to optimize when inferring an observer’s prior. A, A leptokurtotic prior centered on zero formed by a mixture of zero-mean Gaussian components. B, A skewed prior formed by a mixture of Gaussian components with fixed positions and widths. Toolkit script: Fig7_MoGConstrainedFitting.m.

Mixture of Gaussians model fitting to non-Gaussian priors. A, Inferring the shape of a Cauchy prior from a set of 1000 point estimates. Top left, True prior in red and inferred prior in dashed black. Bottom left, The same, but on a semilog axis. Top right, Posteriors for a set of stimuli and measurements, as well x_BLS for each posterior (green line). Bottom right, Set of posteriors and x_BLS inferred from the data using the mixture of Gaussians model. B, Inferring the shape of a bimodal prior from a set of 1000 point estimates. Conventions are the same as in panel A. A slight gamma correction has been applied to the set of posteriors shown in the 2D plots for visibility. Toolkit scripts: Fig8_MoGtoNonGauss.m and Fig8_MoGtoNonGauss2.m for panels A and B, respectively.

Performance of approximations for fitting heavy-tailed priors. A, Diagram illustrating the pipeline for comparing the mixture of Gaussians (MoG) approximation and a single Gaussian (SG) to a full numerical evaluation of two-alternative forced choice data generated with a MoG prior. B, C, Scatter plots illustrate the relationship between the numerical evaluation of the MoG prior model and the SG and approximate MoG approaches. Black circles indicate the points corresponding to the estimated psychometric function values shown in panel A for the SG and MoG approximations. D, Square root of the mean squared error (RMS error) for the MoG analytical approximation and the single Gaussian approximation, summarized over 20 bins of the numerical data. E, Mean signed error distributions for both approximations. Note that axis ranges are set to match Figure 10 for comparison. Toolkit script: Fig9_MoGErrorAnalysis.m.