Elsevier

Behavioural Processes

Volume 114, May 2015, Pages 63-71
Behavioural Processes

Autoshaped choice in artificial neural networks: Implications for behavioral economics and neuroeconomics

https://doi.org/10.1016/j.beproc.2015.01.010Get rights and content

Highlights

  • Autoshaped choice was simulated with an existing neural network model of conditioning.

  • The phenomenon relates to behavioral economics, as it is a form of economic behavior.

  • A theoretical link to neuroeconomics obtains via the model’s neural categories.

  • The phenomenon was simulated without Pavlovian conditioning and response competition.

  • On the methodology of research traditions, the model is being pursued, not accepted.

Abstract

An existing neural network model of conditioning was used to simulate autoshaped choice. In this phenomenon, pigeons first receive an autoshaping procedure with two keylight stimuli X and Y separately paired with food in a forward-delay manner, intermittently for X and continuously for Y. Then pigeons receive unreinforced choice test trials of X and Y concurrently present. Most pigeons choose Y. This preference for a more valuable response alternative is a form of economic behavior that makes the phenomenon relevant to behavioral economics. The phenomenon also suggests a role for Pavlovian contingencies in economic behavior. The model used, in contrast to others, predicts autoshaping and automaintenance, so it is uniquely positioned to predict autoshaped choice. The model also contemplates neural substrates of economic behavior in neuroeconomics, such as dopaminergic and hippocampal systems. A feedforward neural network architecture was designed to simulate a neuroanatomical differentiation between two environment-behavior relations X-R1 and X-R2, where R1 and R2 denote two different emitted responses (not unconditionally elicited by the reward). Networks with this architecture received a training protocol that simulated an autoshaped-choice procedure. Most networks simulated the phenomenon. Implications for behavioral economics and neuroeconomics, limitations, and the issue of model appraisal are discussed.

Introduction

This paper describes the first use of an existing neural network model of conditioning to simulate a phenomenon that is relevant to behavioral economics, the study of the environmental factors that affect economic behavior defined as the allocation of actions across alternatives according to their values. The phenomenon can be called “autoshaped choice” and was first reported by Picker and Poling (1982).

In their basic procedure, they used an autoshaping arrangement for baseline training, where pigeons received randomly intermixed trials of two keylight colors X and Y (e.g., red and yellow), separately paired with food (grain access) in a forward-delay manner, independently of responding. Fifty-percent of X trials and 100% of Y trials was paired with food. Then, pigeons received choice tests where X and Y were presented concurrently and unreinforced. Pigeons tended to choose Y more frequently.

Most features of this phenomenon fit well with behavior-analytic approaches to behavioral economics (e.g., Green and Rachlin, 1975, Hursh, 1980, Hursh, 1984, Kagel et al., 1975, Kagel et al., 1981, Kagel and Winkler, 1972, Lea, 1978). To begin with, the phenomenon involves key pecking, a prototypical emitted, skeletal response widely used in these approaches. This response is emitted in that it is not unconditionally elicited by the reward, at least as demonstrably as typical unconditional responses (e.g., nictitating-membrane responses unconditionally elicited by air puffs or electric shocks to a rabbit's eye). Such approaches view economic behavior as emitted in this sense, rather than unconditionally elicited.

Also, autoshaped choice involves the use of food as a reward. In behavior-analytic approaches to economic behavior, food is analogs to a good or commodity that satisfies some need when consumed. This good occurs more or less frequently, depending on certain conditions. This feature is also defining of autoshaped choice: Food occurs more frequently for one response alternative than another. And choice has been central to behavioral economics as well. In fact, economic behavior is conceived of as a kind of choice behavior.

This feature has encouraged the use of the sort of mathematical models that are used in behavioral studies of choice (e.g., hyperbolic discounting). The use of a neural network model here echoes this use of mathematical models, although, as will become apparent later on, the present model differs from others. In any case, those models describe a phenomenon that has been widely studied in behavioral economics: Preference for a response alternative that is more valuable in virtue of its correlation with a more frequent occurrence of a good. This preference is also observed in autoshaped choice and, as will be shown, is also predicted by the model used here.

Autoshaped choice can thus reasonably be viewed as a form of economic behavior, or at least a simple one. Still, autoshaped choice has a feature that fits less well with behavior-analytic approaches to behavioral economics: The use of Pavlovian contingencies, where the reward is paired with discrete trials of exteroceptive stimuli independently of responding. Behavior-analytic approaches to economic behavior, in contrast, have emphasized operant contingencies, where reinforcement is explicitly scheduled to partly depend on responding and not signaled by discrete trials.

Moreover, in such approaches, the acquisition and maintenance of economic behavior are supposed to require operant contingencies. However, the discovery of autoshaping (Brown and Jenkins, 1968) and automaintenance (Schwartz and Williams, 1972, Williams and Williams, 1969) showed the possibility that emitted responding can be acquired and maintained only through Pavlovian contingencies, without operant contingencies. Despite this, such a possibility has not yet been considered for economic behavior. The present study is the first one to do so.

McSweeney and Bierley (1984) discussed autoshaping in relation to consumer behavior. However, they did not elaborate exactly how autoshaping contributed to economic behavior. Nor did they discuss autoshaped choice. Still, autoshaped choice provides the strongest link thus far between autoshaping and economic behavior. This link implies that the acquisition and maintenance of a simple form of economic behavior do not require operant contingencies. Of course, this does not mean that operant contingencies do not influence the economic behavior. They certainly do. However, autoshaped choice implies that they are not necessary for the acquisition and maintenance of a simple form of economic behavior.

There have been discussions about the influence of Pavlovian contingencies on economic behavior (e.g., Camerer, 2011, Daw and O’Doherty, 2014, Daw and Tobler, 2014, Seymour and Dolan, 2008). However, they still assume that operant contingencies are necessary for the acquisition and maintenance of economic behavior. They have not considered the possibility that economic behavior can be acquired through Pavlovian contingencies alone, without operant contingencies. Autoshaping choice precisely shows this possibility.

The relevance to neuroeconomics is that the present model takes into account brain structures that have been propounded as neural substrates of economic behavior, especially dopaminergic and hippocampal systems (e.g., Camerer et al., 2005, Daw and O’Doherty, 2014, Daw and Tobler, 2014, Dayan and Seymour, 2009, Salamone et al., 2009, Seymour and Dolan, 2008). The model hypothesizes that such systems also underlie autoshaped choice. More specific implications for neuroeconomics will become apparent later. Suffice it to say for now that the model is the first to theoretically link autoshaping, behavioral economics, and neuroeconomics.

The next section gives a general description of the model. Section 3 describes a simulation of autoshaped choice with the model. The paper ends with concluding remarks about some implications for behavioral economics and neuroeconomics, and a philosophical (methodological) issue about explanation in neural-network modeling.

Section snippets

The model: general description

The model has a computational part and a network part. The computational part consists of the activation and learning rules. The network part is a classification of the types of units that can constitute a neural network and guidelines on how to connect them, according to basic general principles of gross vertebrate neuroanatomy. This section focuses on describing the network part in terms of the network architecture that was used for the simulation. The computational part is described in the

A simulation

A simulator designed and coded by the first author was used for the simulation. Nine naive networks with the architecture shown in Fig. 1 received a training protocol that simulated a simplified version of the procedure that Picker and Poling (1982) used in their experiment 1. All networks were naive in that their initial connection weights were low (0.2 for the S'–S″ and S″–H connections, 0.01 for the rest; see Fig. 1).

All networks first received 200 X trials randomly interspersed with 200 Y

General discussion

The results show that the model can simulate autoshaped choice, a simple form of economic behavior, by exploiting the distributed and parallel character of connectionist systems. The distributed character was exploited with an architectural differentiation between two network pathways (S1–S1–M1M1 vs. S2–S2–M2–M2 in Fig. 1) that are separately affected by two environment-behavior relations (X-R1 vs. Y-R2, respectively). Such differentiation allowed the baseline training to promote

Acknowledgments

We thank Lewis Bizo for his invitation to give the talk that resulted in this paper at the 37th Annual Meeting of the Society for Quantitative Analyses of Behavior, and to submit the paper to this issue. We also thank two anonymous reviewers for useful comments to a previous draft. The executable file of the simulator, the necessary files to run the simulations, the source code, and simulation data are available upon request to the first author.

References (56)

  • B. Seymour et al.

    Emotion, decision making, and the amygdala

    Neuron

    (2008)
  • C.P. Bonini

    Simulation of Information and Decision Systems in the Firm

    (1963)
  • P.L. Brown et al.

    Auto-shaping of the pigeon’s key-peck

    J. Exp. Anal. Behav.

    (1968)
  • Burgos, J.E., 1997. Evolving artificial neural networks in Pavlovian environments. In: Donahoe, J.W., Packard-Dorsel,...
  • J.E. Burgos

    Autoshaping and automaintenance: a neural-network approach

    J. Exp. Anal. Behav.

    (2007)
  • J.E. Burgos

    The operant/respondent distinction: a computational neural-network analysis

  • J.E. Burgos et al.

    Structure and function in selectionism: implications for complex behavior

  • K.P. Burnham et al.

    Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach

    (2002)
  • C.F. Camerer

    Psychological influences on economic choice: Pavlovian cuing and emotional regulation

  • C. Camerer et al.

    Neuroeconomics: how neuroscience can inform economics

    J. Econ. Lit.

    (2005)
  • A.C. Catania

    Concurrent performances: inhibition of one response by reinforcement of another

    J. Exp. Anal. Behav.

    (1969)
  • F. Crick et al.

    Are we aware of neural activity in primary visual cortex?

    Nature

    (1995)
  • M. Davison et al.

    Stimulus discriminability, contingency discriminability, and schedule performance

    Anim. Learn. Behav.

    (1985)
  • N.D. Daw et al.

    Multiple systems for value learning

  • M.R.W. Dawson

    Minds and Machines: Connectionism and Psychological Modeling

    (2004)
  • J.W. Donahoe et al.

    A selectionist approach to reinforcement

    J. Exp. Anal. Behav.

    (1993)
  • J.W. Donahoe et al.

    Timing without a timer

    J. Exp. Anal. Behav.

    (1999)
  • J.W. Donahoe et al.

    Theoretical article: behavior analysis and devaluation

    J. Exp. Anal. Behav.

    (2000)
  • Cited by (0)

    View full text