1998 Special IssueMultiple paired forward and inverse models for motor control
Introduction
Humans exhibit an enormous repertoire of motor behavior which enables us to interact with many different objects under a variety of different environments. The ability to perform in such a varying and often uncertain environment is a feature which is conspicuously absent from most robotic control, as robots tend to be designed to operate within rather limited environmental situations. In general, the problem of control can be considered as the computational process of determining the input to some system we wish to control which will achieve some desired output. In human motor control, the problem might be to select the input, i.e. motor command, to achieve some required output, i.e. desired sensory feedback. If we consider an example of lifting a can to ones lips, it may be that the desired output at a specific time is a particular acceleration of the hand as judged by sensory feedback. However, the motor command needed to achieve this acceleration will depend on many variables, both internal and external to the body. Clearly, the motor command depends on the state of the arm, i.e. its joint angles and angular velocities. The dynamic equations governing the system also depend on some relatively unvarying parameters, e.g. masses, moments of inertia, and center of masses of the upper arm and forearm. However, these parameters specific to the arm are insufficient to determine the motor command necessary to produce the desired hand acceleration; knowledge of the interactions with the outside world must also be known. For example, the geometry and inertial properties of the can will alter the arm's dynamics. More global environmental conditions also contribute to the dynamics, e.g. the orientation of the body relative to gravity and the angular acceleration of the torso about the body. As these parameters are not directly linked to the quantities we can measure about the arm, we will consider them as representing the context of the movement. As the context of the movement alters the input–output relationship of the system under control, the motor command must be tailored to take account of the current context.
Considering the number of objects and environments, and their possible combinations, which can influence the dynamics of the arm (let alone the rest of the body), the motor control system must be capable of providing appropriate motor commands for the multitude of distinct contexts that are likely to be experienced. Given the abundance of contexts within which we must act, there are two qualitatively distinct strategies to motor control and learning. The first is to use a single controller which uses all the contextual information in an attempt to produce an appropriate control signal. However, such a controller would demand enormous complexity to allow for all possible scenarios. If this controller were unable to encapsulate all the possible contexts, it would need to adapt every time the context of the movement changed before it could produce appropriate motor commands—this would produce transient and possibly large performance errors. Alternatively, a modular approach can be used in which multiple controllers co-exist, with each controller suitable for one or a small set of contexts. Based on an estimate of the current context, some of the controllers could be activated to generate the appropriate motor command. Under such a modular strategy, there would need to be a context identification process which could choose the appropriate controllers from the set of all possible controllers.
In this paper, we focus on how we learn to produce effective motor control under a variety of contexts. The approach we take is to employ a modular system to tackle the problems of motor learning and control. We will first describe the type of models which must be learned and the benefits which accrue from such modularity. Experimental evidence that supports the use of multiple functionally discrete controllers in humans is briefly reviewed. We then focus on how such modular controllers can be learned and selected, and propose a neural architecture based on multiple paired forward–inverse models. Finally, we describe specific predictions of the model which can be tested experimentally.
Section snippets
Modularity in motor control
In this section, we describe the type of models which must be learned in motor control, i.e. internal forward and inverse models. We then describe the potential benefits of learning these models in a modular fashion.
Do multiple independent controllers exist within the CNS?
Studies of motor adaptation have suggested that we are able to learn multiple controllers and switch between them based on context. In general, when subjects are brought into the laboratory to undergo a motor learning task in which they must adapt to a visual or dynamic perturbation, they can take many movements to adapt (Welch, 1986, Shadmehr and Mussa-Ivaldi, 1994). Although the time course of adaptation can extend over hours, on removal of the perturbation, de-adaptation is often very rapid.
General methodology for multiple modules
Based on the benefits of a modular approach and the experimental evidence for modularity, we propose that the problem of motor learning and control is best solved using multiple controllers, i.e. inverse models. At any one time, one or a subset of these inverse models will contribute to the final motor command. Such a system raises two fundamental computational problems. First, given a set of inverse models which appropriately cover the set of contexts which might be experienced, how is the
Multiple paired forward–inverse model
In this section, we propose a model which can solve the module learning and selection problems in a computationally coherent manner from a single principle. The basic idea of the model is that multiple inverse models exist to control the system and each is augmented with a corresponding forward model. The brain therefore contains multiple pairs of corresponding forward and inverse models. Within each pair, the inverse and forward internal models are tightly coupled both during their acquisition
Physiologically plausible learning
The model described above assumes that the desired motor command is available to train multiple inverse models. This assumption is implausible for biological motor learning, and a more sophisticated physiological computational model involves using the feedback-error-learning model (Kawato et al., 1987, Kawato and Gomi, 1992) with the above simple model.
The total motor command fed to the motor apparatus is the summation of the total
Comparison with other modular systems
We now compare the current model with the previous computational model in which multiple modules can be acquired through learning. Narendra et al., 1995, Narendra and Balakrishnan, 1997have proposed multiple models, each of which describes a different environment, with switching for control. There are several major differences between Narendra's framework and ours, although they seem quite similar at first sight. In their approach, the identification errors for forward models were used to
Model predictions
In this section, we will consider the specific, but as yet untested, predictions of the multiple paired forward–inverse model. We first focus on possible psychophysical experiments before briefly discussing the neurophysiological predictions.
Conclusions
In conclusion, we have presented a new model based on multiple paired forward and inverse models, which is capable of motor learning and control in a modular network. The problem of selecting the appropriate modules is solved by generating a responsibility signal for each module based both on the consequences of performed actions, as estimated by the forward models, and on sensory signals, as estimated by the responsibility predictor. Within each module, the inverse and forward models are
Acknowledgements
We thank Zoubin Ghahramani for helpful discussions on the probabilistic interpretation of the model and Sarah Blakemore for comments on the manuscript. This work was supported by grants from the Wellcome Trust, the Medical Research Council, the Royal Society, the BBSRC and Human Frontier Science Project.
References (48)
- et al.
The cerebellum participates in mental activity
Brain Res.
(1990) - et al.
Recognition of manipulated objects by motor learning with modular architecture networks
Neural Networks
(1993) - et al.
Task decomposition through competition in a modular connectionist architecture: The what and where vision tasks
Cognitive Science
(1991) - et al.
Forward models: Supervised learning with a distal teacher
Cognitive Science
(1992) - et al.
The cerebellum and VOR/OKR learning models
Trends Neurosci.
(1992) - et al.
Theories of motor learning by the cerebellum—reply
Trends Neurosci.
(1993) - et al.
Forward models for physiological motor control
Neural Networks
(1996) Computational approaches to motor control
Trends Cogn. Sci.
(1997)- et al.
Simultaneous opposing adaptive changes in cat vestibulo-ocular reflex directions for two body orientations
Exp. Brain Res.
(1987) - et al.
Consolidation in human motor memory
Nature
(1996)
Maximum likelihood from incomplete data via the EM algorithm
J. Royal Stat. Soc. B
The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads
J. Neurosci.
Motor learning by field approximation
Proc. Natl. Acad. Sci
Modular decomposition in visuomotor learning
Nature
The basal ganglia and adaptive motor control
Science
Gaze-contingent prism adaptation: Optical and motor factors
J. Exp. Psychol.
Separated modules for visuomotor control and learning in the cerebellum: a functional MRI study
NeuroImage
Neurophysiological aspects of the cerebellar motor control system
Int. J. Neurol.
Adaptive mixture of local experts
Neural Computation
Roles of glabrous skin receptors and sensorimotor memory in automatic-control of precision grip when lifting rougher or more slippery objects
Exp. Brain Res.
Cited by (1631)
An ethologically motivated neurobiology of primate visually-guided reach-to-grasp behavior
2024, Current Opinion in NeurobiologyThe cognitive reality monitoring network and theories of consciousness
2024, Neuroscience ResearchThe locus coeruleus as a global model failure system
2024, Trends in NeurosciencesEmotion regulation from an action-control perspective
2023, Neuroscience and Biobehavioral ReviewsInitial development of skill with a reversed bicycle and a case series of experienced riders
2024, Scientific ReportsDisentangling the contribution of individual and social learning processes in human advice-taking behavior
2024, npj Science of Learning