Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems

Abstract

This paper describes how agents can learn an internal model of the world structurally by focusing on the problem of behavior-based articulation. We develop an on-line learning scheme—the so-called mixture of recurrent neural net (RNN) experts—in which a set of RNN modules become self-organized as experts on multiple levels, in order to account for the different categories of sensory-motor flow which the robot experiences. Autonomous switching of activated modules in the lower level actually represents the articulation of the sensory-motor flow. In the meantime, a set of RNNs in the higher level competes to learn the sequences of module switching in the lower level, by which articulation at a further, more abstract level can be achieved. The proposed scheme was examined through simulation experiments involving the navigation learning problem. Our dynamical system analysis clarified the mechanism of the articulation. The possible correspondence between the articulation mechanism and the attention switching mechanism in thalamo-cortical loops is also discussed.

Introduction

How can sensory-motor systems attain an internal representation of the world in structurally organized ways? The consensus in cognitive science and artificial intelligence is that a complex world can be represented efficiently utilizing modular and hierarchical structures of symbol systems (Newell, 1980). However, it is still not understood how such modular and hierarchical representations, if employed, become self-organized in analog neural systems by means of their iterative sensory-motor interactions.

The difficulty lies in the question of “how the continuous sensory-motor flow can be perceived as being articulated into sequences of meaningful representative modules?” Kuniyoshi, Inaba and Inoue (1994) addressed this articulation problem in the robot learning context. In his experiment with an assembling robot, the robot recognizes the various task performances by decomposing them into sequences of modular representations. Subsequently, the robot is able to learn various tasks in terms of combinations of the reusable modular representations obtained. For attaining such a modular representation, the task performance was temporally segmented by means of detecting “meaningful changes” in the observed sensory flow. The problem, however, is that the definitions of these “meaningful changes” were predetermined by designers. Our investigation focuses on how a robot can define “meaningful changes” by itself and perceive a continuous task performance as segmented into reusable modules.

Robot navigation learning, which has a quite long research history, faces the same type of problem. There are basically two types of approach. One is the neural network learning approach. Krose and Eecen (1994), Zimmer (1996) and Nehmzov (1996) showed that for relatively simple workspaces, localization problems for robots can be solved using the topology preserving map scheme (Kohonen, 1982). It is, however, difficult to scale-up this scheme as the very plain representation by a single neural network hardly organizes the modular and hierarchical structure of the learned contents. The other approach is the machine learning approach, used in landmark-based navigation (Kuipers, 1987, Mataric, 1992). In this approach, the travel of the robot is temporally segmented by means of landmarks such as turning at corners, encountering junctions, or going straight along corridors. This temporal segmentation enables the abstraction of robot experiences into a simple chain representation of these landmark types. The scheme can be scaled-up much more readily than the neural network learning approach as the landmarks play the roles of the representative modules. However, the problem is that the landmark types, which are defined by designers, are not necessarily intrinsic to the perceptions of a robot. The representative modules such as corners, junctions, or corridors, if necessary to the problem's solution, ought to be generated from the robot's experiences.

In this paper, we attempt to explain the problems of articulation and structural formation of modules, and hierarchy from the dynamical systems perspective (Beer, 1995, Pollack, 1991, Schoner et al., 1995, Smith and Thelen, 1994, van Gelder, 1999) by focusing on the structural coupling between the internal neural and environmental dynamics. We propose a novel neural architecture, inspired by a modular and hierarchical learning method using neural nets, namely the mixture of experts proposed by Jacobs, Jordan, Nowlan and Hinton (1991). The proposed scheme is examined by conducting simulation experiments of robot navigation learning, where the mechanism of articulation is clarified qualitatively using dynamical systems concepts such as self-organization, coherence and phase transitions. We will discuss briefly the possible correspondence between the mechanism of articulation and the mechanism of attention switching which was proposed to take place in thalamo-cortical loops.