Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems
Introduction
How can sensory-motor systems attain an internal representation of the world in structurally organized ways? The consensus in cognitive science and artificial intelligence is that a complex world can be represented efficiently utilizing modular and hierarchical structures of symbol systems (Newell, 1980). However, it is still not understood how such modular and hierarchical representations, if employed, become self-organized in analog neural systems by means of their iterative sensory-motor interactions.
The difficulty lies in the question of “how the continuous sensory-motor flow can be perceived as being articulated into sequences of meaningful representative modules?” Kuniyoshi, Inaba and Inoue (1994) addressed this articulation problem in the robot learning context. In his experiment with an assembling robot, the robot recognizes the various task performances by decomposing them into sequences of modular representations. Subsequently, the robot is able to learn various tasks in terms of combinations of the reusable modular representations obtained. For attaining such a modular representation, the task performance was temporally segmented by means of detecting “meaningful changes” in the observed sensory flow. The problem, however, is that the definitions of these “meaningful changes” were predetermined by designers. Our investigation focuses on how a robot can define “meaningful changes” by itself and perceive a continuous task performance as segmented into reusable modules.
Robot navigation learning, which has a quite long research history, faces the same type of problem. There are basically two types of approach. One is the neural network learning approach. Krose and Eecen (1994), Zimmer (1996) and Nehmzov (1996) showed that for relatively simple workspaces, localization problems for robots can be solved using the topology preserving map scheme (Kohonen, 1982). It is, however, difficult to scale-up this scheme as the very plain representation by a single neural network hardly organizes the modular and hierarchical structure of the learned contents. The other approach is the machine learning approach, used in landmark-based navigation (Kuipers, 1987, Mataric, 1992). In this approach, the travel of the robot is temporally segmented by means of landmarks such as turning at corners, encountering junctions, or going straight along corridors. This temporal segmentation enables the abstraction of robot experiences into a simple chain representation of these landmark types. The scheme can be scaled-up much more readily than the neural network learning approach as the landmarks play the roles of the representative modules. However, the problem is that the landmark types, which are defined by designers, are not necessarily intrinsic to the perceptions of a robot. The representative modules such as corners, junctions, or corridors, if necessary to the problem's solution, ought to be generated from the robot's experiences.
In this paper, we attempt to explain the problems of articulation and structural formation of modules, and hierarchy from the dynamical systems perspective (Beer, 1995, Pollack, 1991, Schoner et al., 1995, Smith and Thelen, 1994, van Gelder, 1999) by focusing on the structural coupling between the internal neural and environmental dynamics. We propose a novel neural architecture, inspired by a modular and hierarchical learning method using neural nets, namely the mixture of experts proposed by Jacobs, Jordan, Nowlan and Hinton (1991). The proposed scheme is examined by conducting simulation experiments of robot navigation learning, where the mechanism of articulation is clarified qualitatively using dynamical systems concepts such as self-organization, coherence and phase transitions. We will discuss briefly the possible correspondence between the mechanism of articulation and the mechanism of attention switching which was proposed to take place in thalamo-cortical loops.
Section snippets
Prediction learning using sensory-motor flow
The paper introduces robot navigation learning as a prototype problem: our simulation experiments will illustrate how a set of representational primitives or “concepts” emerge and how they enable the construction of “concepts” in the higher level in a dynamic fashion. Our hierarchical learning approach is developed in combination with the prediction learning scheme, which is described below.
Learning to predict the next sensation implies that the system must acquire some analogical model of the
New scheme
Our new proposal in this paper is to use multiple-module RNNs, each of which competes to become an expert at predicting the sensory-motor flow for a specific behavior. The experts achieve their status through learning processes. For example, one module RNN would win in predicting the sensory-motor flow; while the other would win by traveling around a corner and following a straight wall. The switching between the winning RNN modules actually corresponds to the temporal segmentation of the
The environment
The scheme proposed above was investigated in the context of the navigation learning problem by simulation. We assumed a mobile robot with a sensor belt on its forward side holding 20 laser range sensors. The robot, upon perceiving the range image of its surrounding environment, maneuvers in a collision-free manner using a variant of the potential method (Khatib, 1986). (For further details of this maneuvering scheme, see Tani, 1996.)
For our simulations, we adopted two different rooms, namely
On the dynamic mechanism for articulation
We have seen that building blocks for representing specific sensory-motor structures are self-organized in the lower level; the building blocks in the higher level are constructed by combining those in the lower level. The results may be interpreted as being the emergence of internal “symbols”. However, the definition of our “symbols” is quite different to that used in traditional cognitive science studies (Newell, 1980, Newell and Simon, 1976). The “symbols” in our scheme are articulated not
Conclusion
In this paper, we proposed a novel scheme of hierarchical learning for sensory-motor systems using the mixture of RNN experts. The scheme was examined through simulation experiments concerning on-line navigation learning. The results indicate that the robot learns to articulate a continuous sensory-motor flow dynamically, while the modular and hierarchical structures are self-organized internally in a recursive manner across multiple levels. We explained the observed mechanism of articulation
Acknowledgements
The original version of this paper was presented at the International Conference on Simulation of Adaptive Behavior 1998 and later modified for the current publication.
References (39)
A dynamical systems perspective on agent—environment interaction
Artificial Intelligence
(1995)Finding structure in time
Cognitive Science
(1990)Physical symbol systems
Cognitive Science
(1980)- et al.
Dynamics of behavior: theory and applications for autonomous robot architectures
Robotics and Autonomous Systems
(1995) - et al.
Multiple paired forward and inverse models for motor control
Neural Networks
(1998) Robust world-modeling and navigation in a real world
Neuro-Computing
(1996)In the theater of consciousness: the workspace of the mind
(1997)Toward the evolution of dynamical neural networks for minimally cognitive behavior
- et al.
An input–output HMM architecture
- Billard, A. (1996). Do you follow me? or learning to speak through imitation for social robots. Master's thesis,...
Mixtures of controllers for jump linear and non-linear plants
Function of the thalamic reticular complex: the searchlight hypothesis
Proceedings of the National Academy of Sciences, USA
Mode analysis of a ring of a large number of mutually coupled van der Pol oscillators
IEEE Transactions on Circuit Systems
Long short-term memory
Neural Computation
Adaptive mixtures of local experts
Neural Computation
Real-time obstacle avoidance for manipulators and mobile robots
The International Journal of Robotics Research
Self-organized formation of topographically correct feature maps
Biological Cybernetics
A self-organizing representation of sensor space for mobile robot navigation
Proceedings of the International Conference on Intelligent Robotics and Systems
A qualitative approach to robot exploration and map learning
Proceedings of AAAI Workshop Spatial Reasoning and Multi-Sensor Fusion
Cited by (172)
Where to from here? On the future development of autonomous vehicles from a cognitive systems perspective
2022, Cognitive Systems ResearchHierarchical generative modelling for autonomous robots
2023, Nature Machine Intelligence