Elsevier

Journal of Physiology-Paris

Volume 106, Issues 5–6, September–December 2012, Pages 297-315
Journal of Physiology-Paris

Local field potentials and border ownership: A conjecture about computation in visual cortex

https://doi.org/10.1016/j.jphysparis.2012.08.001Get rights and content

Abstract

Border ownership is an intermediate-level visual task: it must integrate (upward flowing) image information about edges with (downward flowing) shape information. This highlights the familiar local-to-global aspect of border formation (linking of edge elements to form contours) with the much less studied global-to-local aspect (which edge elements form part of the same shape). To address this task we show how to incorporate certain high-level notions of distance and geometric arrangement into a form that can influence image-based edge information. The center of the argument is a reaction—diffusion equation that reveals how (global) aspects of the distance map (that is, shape) can be “read out” locally, suggesting a solution to the border ownership problem. Since the reaction—diffusion equation defines a field, a possible information processing role for the local field potential can be defined. We argue that such fields also underlie the Gestalt notion of closure, especially when it is refined using modern experimental techniques. An important implication of this theoretical argument is that, if true, then network modeling must be extended to include the substrate surrounding spiking neurons, including glia.

Highlights

► Border ownership is a Gestalt property of figures not backgrounds. ► Neurons in the visual cortex exhibit a border-ownership response. ► We develop a novel computational model of border ownership involving local field potentials. ► Results show how distant boundary influences are represented in the field.

Introduction

One community of researchers in computational neuroscience seeks to understand how spiking neural networks can explain the remarkable facility that brains have for solving information processing tasks. This community is focussed on tasks such as color representation, object recognition, grasping and navigation. The activity of individual neurons and the small scale circuits they form are central. We shall refer to this as Community I. At the same time there is another community, Community II, that has been interested in large-scale phenomena, for example how the rhythms revealed in cortical potentials (e.g., the EEG) can signal various brain states (e.g., Haider and McCormick, 2009), sleep rhythms (e.g., Steriade et al., 1993) and the motor coordination required for swimming (e.g., Olufsen et al., 2003). Central to the thinking about brain rhythms are fields and potentials.

At a biophysical level these communities must converge. The potentials have a source and dendritic and neuronal ionic activity are at the heart of it. But the issue is less clear at more abstract levels. There are examples in which researchers have attempted to cross between communities, using techniques from one to inform solutions to problems from the other. For example, feature maps (at the center of Community I) located in different cortical areas provide distributed codes for structure; gamma rhythms (community II) have been suggested as the mechanism for coordinating them; i.e., as a solution to the “binding problem” (see review in Buzsaki, 2006). Following Buzsaki’s metaphor, these rhythms define a kind of clock, like the reference provided by the conductor in an orchestra. But if such rhythms are viewed as a conductor, providing the means by which distant players remain orderly and harmonious, what is the music they are playing? What is the information underlying the information processing tasks?

W. Kohler, one of the founders of Gestalt psychology (Kohler and Held, 1949) was among the first to attempt a concrete, large-scale explanation of information processing tasks in terms of cortical potentials. Working with surface electrodes, he sought to relate EEG potentials to the direction of motion in visual stimuli. Lashley et al. (1951), however, showed that this gross association was infeasible. This and other rebuttals cast doubt on whether Community II could directly solve information processing problems. To push matters even further, the many subsequent discoveries about spiking neurons and the networks they form established information processing as a task well within Community I. Much of computational neuroscience is now organized around understanding the information processing capabilities of these networks and how large-scale cortical potentials can be derived from them; for an encyclopaedic review, see Wang (2010). Although now largely out of fashion (although there are exceptions; see below), the early Gestalt theorists did establish the idea of relating perceptual phenomena to cortical potentials. In this paper we shall develop a conjecture that takes this one step further, by showing that field effects could, at least in principle, play a fundamental information processing role.

Another major contribution of Gestalt psychology were the demonstrations of perceptual organization (Wertheimer, 1923), such as good continuation and common fate. These were in principle still thought to be related to brain function via physical analogies such as energy minimization and variational principles. For an extensive review of a century of Gestalt contributions in this area, see Wagemans et al. (in press); for a modern view of related physical principles see Bressloff (2012).

The approach and the focus on the whole – the Gestalt – precluded a focus on the component parts, which in computational theories are fundamental. One area in which there has been a revival of certain Gestalt ideas that do involve component organization is in visual neuroscience. The classical representation of local processing—the receptive field—is being enlarged to include multiple constituents: those that derive directly from feedforward processing; those that derive from lateral (horizontal) connections; and those that derive from feedback processing (Angelucci et al., 2002). Developments in imaging help enormously here, because now the total synaptic integration field (or the non-classical receptive field) can be described in operational terms. This level of experimental resolution has allowed Chavane et al. (2000) to cleverly relate these extra-classical components of a neuron’s discharge field to apparent motion effects, thus opening a possible connection between cortical field potentials and Gestalt phenomena that was not available to Kohler and colleagues a half century ago. It is a related issue that we address in this paper – how local field potentials could play a direct role in information processing – but from a more abstract perspective.

The heart of the matter is to determine whether there are information processing roles for field potentials that are not naturally placed within spiking networks alone? That is to ask: Is there a direct Community II solution for a community I problem; and can these be specified abstractly? In this paper we provide evidence that the answer is affirmative; that fields and potentials can play a fundamental – and direct – information processing role in an unexpected manner. If true neurobiologically, our conjecture could have implications for the receptive field concept, for neural modeling, and for understanding neural computation more generally. But we stress that this paper is highly conjectural, and much remains to be done before it can be confirmed. We place the conjecture in the literature to stimulate discussion, following in the spirit of Olshausen and Fields’s essay: “What Is the Other 85 Percent of V1 Doing?” (Olshausen and Field, 2006).

To formulate our conjecture we switch from general considerations to a more focussed one. We ask, in particular, how distant information might influence local processing. This arises specifically in the Gestalt notions of figure and ground (Koffka, 1935), and considering this question in some detail allows us to make direct computations. Asserting that figure/ground is simply coordinated activity, as in rhythms coordinating processes across different areas, does not explain the Gestalt observations; somehow certain information about boundaries and their arrangements must be integrated over long distances. Precisely which information is involved and how it is processed is not well understood. Nor are the relationships with traditional network models. This is why it is helpful to address these questions mathematically.

The crux of the matter is to explain how general boundary distributions at a distance can effect local boundary signals at a point. Local fields could play a role in this signaling, and we provide an abstract characterization of how this might occur. The result is a conjecture about how long-distance activity (a cell assembly in a higher area) can influence local activity (in a lower area) embedded in a local field (derived from feedback) using a phase-of-firing code.

Our argument will integrate ideas from local field potentials, border responses, visual psychophysics and phase codes into a conjecture about how certain Gestalts can be computed. Because so few data are available in detail, the argument hinges on a mathematical idea expressed as a computation and its implications. In short, we claim that the local field potential deriving from feedback processes can directly encode general information about distant boundaries and their arrangement in a way that smooths over specifics. To motivate it, we first review material from several different areas. We then develop it from a more abstract perspective.

We begin with the concept of closure. In computer vision it is now commonplace to seek image “segmentations” that delimit figures by wrapping a boundary around them. That is, a figure is a set of image pixels separate from the background; this set is normally taken to be connected. (Think of an image of a person, the figure, in front of a wall of books, the background.) Thus the Jordan curve theorem applies, and boundaries around such figures are defined by closed curves. People can draw such curves, if instructed to do so, and operationally a database of human drawn segmentations serves as a widely used benchmark against which computer vision segmentations are evaluated (Martin et al., 2001). But we know from Gestalt psychology and from neuroscience that this goal for a segmentation is neither natural nor complete. First, the visual system does not organize pixels; it computes a series of more abstract representations in e.g. (position, orientation) space that involve boundaries. But these boundaries need not be closed (e.g., boundaries ending at cusp points, Lawlor et al., 2009). Second, regarding naturalness, figures need not have a fully closed boundary. (Now, think of the person’s hand as the figure and her torso as background; where is the closure of the hand?)

Gestalt psychologists have refined the notion of figure/ground organization beyond closed curves in two subtle but important ways. They were the first to show that boundaries need not be closed, continuous, or even connected. This suggests relaxing the rigid inside/outside topology sought in computer vision to something like an enclosure field distributed over a region. Understanding the basic properties of such fields provides a rough glimpse of where we are heading. Second, the Gestalt psychologists showed that a key property of the concept of figure is that it “owns” the boundary. Explaining how this—border ownership—could be computed neurobiologically is exactly where we are heading.

We propose that enclosure fields and border ownership are intimately related concepts in vision. That there are rich interactions between them is easily demonstrated; see Fig. 1. To develop a computational conjecture on how these might be computed, we first develop a more traditional approach to computing borders. This is a standard view of how local (‘edge’) measurements could be integrated into boundary fragments by a process of good continuation. This aspect of border continuation is essentially a neighborhood computation and could be implemented by long-range horizontal connections. In our view it relies on relatively precise geometric relationships (how boundary fragments can be integrated via curvature).

The border-ownership computation is different: it can involve integrating information from much greater distances and across many different (boundary-specific) geometries. The challenge is not to determine how a given local edge element can be owned by a specific figure—this can be learned or pre-wired—but rather to determine how a given local edge element can be owned by many different figures in different contexts. In neurobiological terms, many different cell assemblies can be representing the possible figure completions for a given edge cell: how can all of these putatively different cell assemblies be integrated into a single signal? Somehow global shape considerations must come into play in a more general manner than (local) differential geometry. In effect there is an “information at a distance” problem: how do distant edges “inform” whether a given edge is part of the figure or part of the background, regardless of the exact shape of the figure?

These ideas are all reviewed in the next sections. We then begin to develop our contribution, which is a partial differential equation that characterizes those aspects of the “information at a distance” property that are inherent in border ownership. Several of its relevant properties are illustrated, and to demonstrate its usefulness concretely we briefly review an application of these ideas to recognizing complex features such as airports.

At the heart of the matter is the computation of distance-like measures, which find a very natural home in the local field potential. Perhaps the most controversial—and the most exciting—aspect of our conjecture is the manner in which it shows how the information at a distance for border ownership can be integrated with the border continuation computations; in effect, we suggest that a phase-of-firing code could unify these two separate tasks.

The explicit steps in our argument, and an outline of the paper, are as follows:

  • 1.

    Borders and figures are related.

    • (a)

      Contour integration is a semi-local process (Section 2.1). It is accomplished in parallel, in which arrangements of edge elements that are coherent; i.e., co-circular, define circuitry in V1. It could involve long-range horizontal connections.

    • (b)

      Longer contour fragments derive from the projection of co-circular edge elements into higher visual areas.

    • (c)

      Consistent border fragments define figures (Section 2.2). This might be thought of as the feedforward part of the computation. It is demonstrated psychophysically and leads to axis-based shape descriptors. The distance map lies at the foundation of these representations.

    • (d)

      Figures define border fragments (Section 2.3). This might be thought of as the feedback part of the computation. This is where the border ownership responses are defined and the physiological experiments described. It also is demonstrated psychophysically.

    • (e)

      Existing models of border ownership are described (Section 2.5). The basic question of how global, distant edge information can influence local edge information is raised.

    • (f)

      Boundaries need not be connected or closed to provide a border ownership signal (Section 2.6). This suggests more of a field-like computation.

  • 2.

    The enclosure field computation is carried by the local field potential.

    • (a)

      Motivation from plant biology leads to a first differential equation (Section 3). This is just a warm-up, to appreciate how this kind of model can arise.

    • (b)

      The relevant edge-based model is then developed (Section 3.2). This is the basic differential equation underlying our model. It shows how the equilibrium distribution can carry signals about distant edge arrangements; i.e., information about the distance map. This is where integration over different shape completions is accomplished.

  • 3.

    The substrate for the enclosure field computation.

    • (a)

      Local field potentials are reviewed (Section 4.1). The question of whether there is more to them than just blurred neuronal activity is key. We propose that the lfp carries the information about the distance map and the enclosure field.

    • (b)

      Cortical tissue is rich in glial cells, which could also support the development of fields (Section 5.2). Why the astrocytes have channels that could be used for information processing is raised. Perhaps they assist in developing the enclosure field.

    • (c)

      Phase of firing codes could then integrate the local field context with the border responses (Section 4.2).

    • (d)

      Conjecture: the arrangement of (boundary-based) neural activity gives rise to a component of the local field potential that modulates border-ownership neurons (Section 4.3).

Each of these steps is developed in a subsequent section.

Section snippets

Border and figural interactions

In this background Section we lay out three basic aspects of our problem. First, some borders can be inferred from images. This is a classical problem, it can be formulated in geometric terms, and we illustrate one approach that relates the geometric inferences to neurobiology. Second, borders can define shapes. When the borders are closed this is essentially an application of the Jordan curve theorem: there is a well-defined set of locations that defines the inside and another that defines the

Global distance information signaled locally

To review, the key challenge behind this paper is to determine how to represent the effects of distant events—sections of a figural boundary— locally and in a fashion that could be realized relatively early in the visual system. This amounts to representing aspects of the distance map so that they can be used in an intermediate-level manner. Thus a middle ground is sought between abstract high-level representations such as complete surface and occlusion representations and/or complete shape

The enclosure field conjecture

The plant model just reviewed suggests a very different tack for approaching border ownership. To explore this, we build on the idea of signals phrased in terms of the equilibria of a differential equation in a physical medium. We enlarge the focus from spiking neurons, as in the previous models of border ownership (Fig. 5), to include the substrate in which the spiking neurons are embedded. This medium supports a rich electric potential and local field.

Our conjecture, in summary terms, is that

Implications of the model

The general message of the model conjectured in this paper is that neural computations emerge not only from networks of spiking neurons, but also from the physical substrate in which they are embedded. This elaborates current thinking about neuronal anatomy, which thus far has focused on wiring length minimization at a large scale (Klyachko and Stevens, 2003) and on wiring-length and volume exclusion at a small scale (Mitchison, 1991, Cherniak, 1992, Chklovskii et al., 2002). Thus far

Summary and conclusions

The local field potential is normally thought of as a facilitator for long-distance communication and coordination. We argued that there could be a more direct information processing role to this, and concentrated on the border-ownership computation to make the argument concrete.

Two lines were essential. First, we argued that border ownership requires a field-like realization of enclosure, and provided psychophysical evidence in support of this. Second, we developed a structural coupling

Acknowledgements

I thank Pavel Dimitrov and Matthew Lawlor for their fundamental contributions. For constructive discussions plus references and comments, I thank John Allman, Andreas Andreau, Gyorgy Buzsaki, Jacob Feldman, Yves Fregnac, Rudiger von der Heydt, Naoki Kogo, Terrence Sejnowski, Robert Shapley, Manish Singh, Michael Stryker, John Tsotsos and Johan Wagemans. Research supported by AFOSR, ARO, NIH and NSF.

References (100)

  • B. Haider et al.

    Rapid neocortical dynamics: cellular and network mechanisms

    Neuron

    (2009)
  • C.C. Hung et al.

    Medial axis shape coding in macaque inferotemporal cortex

    Neuron

    (2012)
  • S. Katzner et al.

    Local origin of field potentials in visual cortex

    Neuron

    (2009)
  • C. Kayser et al.

    Sensory neural codes using multiplexed temporal scales

    Neuron

    (2009)
  • B. Kimia

    On the role of medial geometry in human vision

    J. Physiol. (Paris)

    (2003)
  • P. Latham et al.

    Phase coding: spikes get a boost from local fields

    Curr. Biol.

    (2008)
  • M. Lawlor et al.

    Boundaries, shading, and border ownership: a cusp at their interaction

    J. Physiol. (Paris)

    (2009)
  • T.S. Lee et al.

    The role of the primary visual cortex in higher level vision

    Vis. Res.

    (1998)
  • M.A. Montemurro et al.

    Phase-of-firing coding of natural visual stimuli in primary visual cortex

    Curr. Biol.

    (2008)
  • M. Nedergaard et al.

    New roles for astrocytes: redefining the functional architecture of the brain

    Trends Neurosci.

    (2003)
  • A. Nimmerjahn et al.

    Motor behavior activates Bergmann glial networks

    Neuron

    (2009)
  • S. Panzeri1 et al.

    Sensory neural codes using multiplexed temporal scales

    Trends Neurosci.

    (2010)
  • M. Paradiso et al.

    Lightness, filling-in, and the fundamental role of context in visual perception

    Prog. Brain Res.

    (2006)
  • F. Qiu et al.

    Figure and ground in the visual cortex: V2 combines stereoscopic cues with gestalt rules

    Neuron

    (2005)
  • B. Ransom et al.

    New roles for astrocytes (stars at last)

    Trends Neurosci.

    (2003)
  • D.R. Simmons et al.

    Vision in autism spectrum disorders

    Vis. Res.

    (2009)
  • A. Stepanyants et al.

    Neurogeometry and potential synaptic connectivity

    Trends Neurosci.

    (2005)
  • A. Treisman et al.

    A feature-integration theory of attention

    Cogn. Psychol.

    (1980)
  • S. Ullman

    Visual routines

  • L. Zhaoping

    Border ownership from intracortical interactions in visual area v2

    Neuron

    (2005)
  • S.W. Zucker et al.

    Toward a low-level description of dot clusters: labelling edge, interior, and noise points

    Comput. Graph. Image Process.

    (1979)
  • C.A. Anastassiou et al.

    The effect of spatially inhomogeneous extracellular electric fields on neurons

    J. Neurosci.

    (2010)
  • A. Angelucci et al.

    Circuits for local and global signal integration in primary visual cortex

    J. Neurosci.

    (2002)
  • A. Araque et al.

    Glial cells in neuronal network function

    Philos. Trans. R. Soc. B

    (2010)
  • A.M. Bartlett et al.

    Saccades during object viewing modulate oscillatory phase in the superior temporal sulcus

    J. Neurosci.

    (2011)
  • O. Ben-Shahar et al.

    Geometrical computations explain projection patterns of long-range horizontal connections in visual cortex

    Neur. Comput.

    (2003)
  • P.C. Bressloff

    Spatiotemporal dynamics of continuum neural fields

    J. Phys. A: Math. Theor.

    (2012)
  • G. Buzsaki

    Rhythms of the Brain

    (2006)
  • G. Buzski et al.

    The origin of extracellular fields and currents eeg, ecog, lfp and spikes

    Nat. Rev. Neurosci.

    (2012)
  • L. Calabi et al.

    Shape recognition, prairie fires, convex deficiencies and skeletons

    Am. Math. Monthly

    (1968)
  • C. Cherniak

    Local optimization of neuron arbors

    Biol. Cybern.

    (1992)
  • H.I. Choi et al.

    Mathematical theory of medial axis transform

    Pacific J. Math.

    (1997)
  • P.S. Churchland et al.

    A critique of pure vision

  • E. Craft et al.

    A neural model of figure-ground organization

    J. Neurophysiol.

    (2007)
  • P. Dimitrov et al.

    A constant production hypothesis that predicts the dynamics of leaf venation patterning

    Proc. Nat. Acad. Sci. (USA)

    (2006)
  • Dimitrov, P., Zucker, S.W., 2009. Patterns in plant development #1: Uniform production and proportional destruction of...
  • Dimitrov, P., Zucker, S.W., 2009. Patterns in plant development #2: Facilitated transport and uniform gradient,...
  • Dimitrov, P., Lawlor, M., Zucker, S., 2011. Distance images and intermediate-level vision. In: Third International...
  • B. Dubuc et al.

    Complexity, confusion, and perceptual grouping. Part i: the curve like representation

    Int. J. Comput. Vis.

    (2001)
  • B. Dubuc et al.

    Complexity, confusion, and perceptual grouping. Part ii: mapping complexity

    Int. J. Comput. Vis.

    (2001)
  • Related ideas were developed for application in computer vision (Dimitrov et al., 2011).

    View full text