Learning an Efficient Hippocampal Place Map from Entorhinal Inputs Using Non-Negative Sparse Coding

Abstract Cells in the entorhinal cortex (EC) contain rich spatial information and project strongly to the hippocampus where a cognitive map is supposedly created. These cells range from cells with structured spatial selectivity, such as grid cells in the medial EC (MEC) that are selective to an array of spatial locations that form a hexagonal grid, to weakly spatial cells, such as non-grid cells in the MEC and lateral EC (LEC) that contain spatial information but have no structured spatial selectivity. However, in a small environment, place cells in the hippocampus are generally selective to a single location of the environment, while granule cells in the dentate gyrus of the hippocampus have multiple discrete firing locations but lack spatial periodicity. Given the anatomic connection from the EC to the hippocampus, how the hippocampus retrieves information from upstream EC remains unclear. Here, we propose a unified learning model that can describe the spatial tuning properties of both hippocampal place cells and dentate gyrus granule cells based on non-negative sparse coding from EC inputs. Sparse coding plays an important role in many cortical areas and is proposed here to have a key role in the hippocampus. Our results show that the hexagonal patterns of MEC grid cells with various orientations, grid spacings and phases are necessary for the model to learn different place cells that efficiently tile the entire spatial environment. However, if there is a lack of diversity in any grid parameters or a lack of hippocampal cells in the network, this will lead to the emergence of hippocampal cells that have multiple firing locations. More surprisingly, the model can also learn hippocampal place cells even when weakly spatial cells, instead of grid cells, are used as the input to the hippocampus. This work suggests that sparse coding may be one of the underlying organizing principles for the navigational system of the brain.

in Franzius et al. (2007b)). Additionally, the weights connecting grid and place cells can be 77 positive or negative, and the place cell responses were manually shifted by a constant term to be kept non-78 negative, which puts into question the biological realisation of the model. Furthermore, previous models do 79 not investigate how well the learned hippocampal place map represents the entire spatial environment and 80 how the weakly spatial cells in the EC can contribute to the formation of the place map. 81 Sparse coding (Olshausen and Field, 1996) provides a compelling explanation of many experimental find-82 ings of brain network structures. One particular variant, non-negative sparse coding (Hoyer, 2003), has 83 recently been shown to account for a wide range of neuronal responses in brain areas (see Beyeler et al. 84 (2019) for a review). However, whether sparse coding can learn the hippocampal place map has not previ-85 ously been investigated. 86 Here we apply sparse coding with non-negative constraint, where neuronal responses and connection 87 weights are restricted to be non-negative, to building a learning model of hippocampal cells using EC in-88 puts. Our results show that, when grid cells are used as the entorhinal input, single-location hippocampal 89 place cells that tile the entire environment can be learned, given the sufficient diversity in grid parameters. 90 However, if there is a lack of diversity in any grid parameters, the learning of the hippocampal place cells is where E( r) is the grid cell response at the spatial location r = (x, y), λ is the grid spacing, θ is the grid 119 orientation, r 0 = (x 0 , y 0 ) represents the grid phase, and u j = (cos(2πj/3 + θ)), sin(2πj/3 + θ)) is the 120 unit vector with direction 2πj/3 + θ. E(·), described in Equation 1, is normalised to have a maximal 121 value of 1 and minimum of 0. Because of the periodicity of the hexagonal pattern, the grid orientation, θ, 122 lies in the interval of [0, π/3), and the phases in both x and y axes are smaller than the grid spacing; i.e., 123 0 ≤ x 0 , y 0 < λ. 124 Since grid cells have different spacings, orientations and phases, Equation 1 is used to generate diverse grid 125 cells. The value of the grid spacing, λ, ranges in value from 28 cm (Hafting et al., 2005;Solstad et al., 2006) 126 and increases by a geometric ratio 1.42 that is consistent with experimental results (Stensola et al., 2012) 127 and the optimal grid scale derived by a mathematical study (Wei et al., 2015). x-axis, they will have the values x 0 = 0 and λ/2. The resulting total number of modelled entorhinal cells 135 (grid cells), denoted as N e , will be the product of numbers of spacings, orientations and phases: Some examples of grid cells described by Equation 1 are shown in Figure 1D. These grid cells have diverse   The firing field of each grid cell is modelled as the sum of multiple grid fields whose centres are located 164 at the vertices of the hexagonal grid. The grid field at vertex (x v , y v ) is described by a function with the 165 following form (Neher et al., 2017) 166 where γ v is the amplitude, σ determines the radius of the grid field, and the response will be γ v /5 at a 167 distance σ away from the centre. σ is determined by the grid spacing, λ, with σ = 0.32λ (Neher et al.,

182
Weakly spatial cells are used as another type of entorhinal input to investigate how they contribute to the 183 formation of the hippocampal place map.

184
Sparse coding with non-negative constraint 185 Sparse coding was originally proposed by Olshausen and Field (1996) to demonstrate that simple cells in 186 the primary visual cortex represent their sensory input using an efficient neuronal representation, namely 187 that their firing rates in response to natural images tend to be sparse (rarely attain large values) and statis-188 tically independent. In addition, sparse coding finds a reconstruction of the sensory input through a linear 189 representation of features with minimal error, which can be understood as minimising the following cost respectively. Q (·) is the derivative of Q(·), the dot notation represents differentiation with regard to time, 203 and · is the average operation. shows that the dynamics in Equation 5 can be implemented via thresholding and local competition in neural 206 circuits, as described by where 1 is the identity matrix, τ is the time constant, u can be interpreted as the membrane potential, and β where τ is the time constant for modelled hippocampal cells, β is the threshold of the rectifying function of where η is the learning rate. Elements of A are kept non-negative during training; i.e., the element will be after training, we use the method of reverse correlation to recover the firing fields, denoted as F, of modelled 276 hippocampal cells. We present K uniformly sampled random locations, r 1 , · · · , r K , to the model, compute 277 according to Equation 8 the neural responses of a modelled hippocampal cell, s 1 , · · · , s K , and then compute 278 the firing field, F, of this modelled hippocampal cell by K = 10 5 is used in this paper.

13
Fitting firing fields to functions 281 In order to obtain the centre and size of the firing field of the modelled hippocampal cell, F is fitted by a 282 function Q(x, y) of the form where γ is the amplitude, σ is the breadth of the firing field, and (x c , y c ) represents the centre of the function.

284
The built-in MATLAB (version R2020a) function, lsqcurvefit, is used to fit these parameters. The fitting 285 error is defined as the square of the ratio between the fitting residual and firing field. After fitting, the fitting for the firing field: (1) the fitting error is smaller than 15% (2) the breadth, σ, is larger than 5 cm. These two 293 rules exclude any modelled hippocampal cells with no obvious firing field or with multiple-location firing 294 field. The firing field of a place cell is called place field. 295 Measuring the uniformity of place cell representation 296 For place cells that meets the criteria defined above, the field centre (x c , y c ) fitted by Equation 11 indicates 297 the spatial location that the place cell responds to. We measured how well all place cells represent the entire 298 environment using two measures.

299
The first measure is distance to place field, d PF , which indicates the Euclidean distance between each spatial 300 location (p x , p y ) in the environment and the nearest place field, described as If the distance to a place field is large for a location, it means that there are no place fields near this location.

302
Therefore, the distribution of this measure can tell us how well place fields of all place cells tile the entire 303 spatial environment. When all spatial locations have small values of d PF , the entire environment is tiled by 304 the place cells.

305
The second measure is nearest distance, d ND . We define d ND of place cell j as the maximal Euclidean 306 distance of 2 nearest centres, described as where (x j , y j ) is the centre of the place field, min 2 returns a set of 2 smallest values. The distribution of d ND that evenly tile the 1m × 1m environment, the nearest distance will be 100/(10 − 1) ≈ 11.11 cm for each 316 place cell and the distance to place field for every location is smaller than 11.11/2 ≈ 5.56 cm.

318
When diverse grid cells are used as the entorhinal input, the model can learn place cells that The centres of all the place cells are displayed together in the 1m × 1m spatial environment represented by 332 a 32 × 32 pixel-like image, Figure 3B, which shows that the centres of the 100 place cells tile the entire 333 environment without any overlap. In addition, the box plot in Figure 3C shows that any location within the 334 space is within a distance of no more than 8.2 cm from the nearest place fields. The histogram of nearest distance of all 100 place cells is displayed in Figure 3D, which shows that the distribution is centred around  The competition introduced by sparse coding provides the inhibition for place cells

350
The connectivity profile between 600 grid cells and 100 place cells is plotted in Figure 4A, which shows 351 that each place cell selects a group of particular grid cells with different weights. As a result, the overall 352 feedforward connection from the spatial environment to the place cells, namely the matrix product EA, 353 has the spatial structure plotted in Figure 4B, which shows that each place cell is selective to one spatial 354 location similar to the firing fields ( Figure 3A). However, EA has strong average offsets, which can be seen 355 from the grey background in Figure 4B.      Figure 10A shows the 18 learned place cells. Figure 10C shows that these place cells have radius from 18.71 spatial pattern to the fully developed hexagonal grid pattern.

475
In this section, 600 weakly spatial cells are used as the entorhinal input to the hippocampus and there are 476 100 modelled hippocampal cells. We use a smaller learning rate (η = 0.01) and more epochs (30,000) for 477 the learning here. The firing fields of ten example weakly spatial cells are shown in Figure 11A.  Compared with Figure 3 and 8A, using weakly spatial cells instead of grid cells results in learning a hip-485 pocampal place-map with less resolution. The mean radius of place fields using weakly spatial cells as 486 the entorhinal input ( Figure 11B: mean 11.45 cm with standard deviation 2.14 cm) is larger than Figure 3 487 (8.92 cm) and Figure 8A (8.75 cm). Furthermore, the nearest distance in Figure 11E Figure 11E suggests that the irregular fields of weakly spatial cells lead to the 491 less even tiling of place cells. However, the learned place map still covers the entire environment well with 492 small distance to place field ( Figure 11D) and efficiently with a hexagonal lattice ( Figure 11C).

493
The learned place map is robust to noise even when weakly spatial cells are used 494 Furthermore, the model is quite robust to noise and an efficient place map can still be learned, even though 495 a relatively strong noise is added to the modelled entorhinal cell responses in Equation 8: where n is the Gaussian noise with mean 0 and variance 1, and γ n is the amplitude of the noise. Note that 497 the maximal value of E T r is 1 because E is normalised to have the maximum 1. We find that the model can 498 still learn an efficient map when γ n is 0.3 ( Figure 12). Though there are places in the environment not covered by place cells, generally the modelled hippocampal 500 cells learn place fields that efficiently tile the entire environment.

501
Above all, the model is consistent with the experimental evidence that place cells emerge earlier than grid cells during development and a possible explanation is that the neural system can learn a hippocampal 503 map even when the hexagonal spatial field is not well developed, and place cells can maintain their place 504 fields when grid cells are inactivated because weakly spatial cells in the EC can lead to the emergence of a 505 hippocampal place map.

506
Sparse coding can learn hippocampal place cells even though the input cells from the EC are weakly tuned 507 to the spatial environment. Thus, input cells with stronger spatial selectivity can provide more spatial in-508 formation so that unique place field can be decoded by sparse coding. Barry  of how sparse coding can be implemented is still not very clear, which is discussed in the next section.

656
Underlying neural circuits 657 Our study examines the extent to which sparse coding is as an underlying principle in the navigational sys-658 tem of the brain. However, the current model implies no specific neural circuits for the implementation of input to dorsal hippocampus. Science 308:1792-1794.