Untangling invariant object recognition

Despite tremendous variation in the appearance of visual objects, primates can recognize a multitude of objects, each in a fraction of a second, with no apparent effort. However, the brain mechanisms that enable this fundamental ability are not understood. Drawing on ideas from neurophysiology and computation, we present a graphical perspective on the key computational challenges of object recognition, and argue that the format of neuronal population representation and a property that we term ‘object tangling’ are central. We use this perspective to show that the primate ventral visual processing stream achieves a particularly effective solution in which single-neuron invariance is not the goal. Finally, we speculate on the key neuronal mechanisms that could enable this solution, which, if understood, would have far-reaching implications for cognitive neuroscience.

Introduction

Our daily activities rely heavily on the accurate and rapid identification of objects in our visual environment. The apparent ease of with which we recognize objects belies the magnitude of this feat: we effortlessly recognize objects from among tens of thousands of possibilities and we do so within a fraction of a second, in spite of tremendous variation in the appearance of each one. Understanding the brain mechanisms that underlie this ability would be a landmark achievement in neuroscience.

Object recognition is computationally difficult for many reasons, but the most fundamental is that any individual object can produce an infinite set of different images on the retina, due to variation in object position, scale, pose and illumination, and the presence of visual clutter (e.g. 1, 2, 3, 4, 5). Indeed, although we typically see an object many times, we effectively never see the same exact image on our retina twice. Although several computational efforts have attacked this so-called ‘invariance problem’ (e.g. 1, 3, 6, 7, 8, 9, 10, 11, 12), a robust, real-world machine solution still evades us and we lack a satisfying understanding of how the problem is solved by the brain. We believe that these two achievements will be accomplished nearly simultaneously by an approach that takes into account both the computational issues and the biological clues and constraints.

Because it is easy to get lost in the sea of previous studies and ideas, the goal of this manuscript is to clear the table, bring forth key ideas in the context of the primate brain, and pull those threads together into a coherent framework. Below, we use a graphical perspective to provide intuition about the object recognition problem, show that the primate ventral visual processing stream produces a particularly effective solution in the inferotemporal (IT) cortex, and speculate on how the ventral visual stream approaches the problem. Along the way, we argue that some approaches are only tangential to, or even distract from, understanding object recognition.