Traces of Semantization, from Episodic to Semantic Memory in a Spiking Cortical Network Model

Abstract Episodic memory is a recollection of past personal experiences associated with particular times and places. This kind of memory is commonly subject to loss of contextual information or “semantization,” which gradually decouples the encoded memory items from their associated contexts while transforming them into semantic or gist-like representations. Novel extensions to the classical Remember/Know (R/K) behavioral paradigm attribute the loss of episodicity to multiple exposures of an item in different contexts. Despite recent advancements explaining semantization at a behavioral level, the underlying neural mechanisms remain poorly understood. In this study, we suggest and evaluate a novel hypothesis proposing that Bayesian–Hebbian synaptic plasticity mechanisms might cause semantization of episodic memory. We implement a cortical spiking neural network model with a Bayesian–Hebbian learning rule called Bayesian Confidence Propagation Neural Network (BCPNN), which captures the semantization phenomenon and offers a mechanistic explanation for it. Encoding items across multiple contexts leads to item-context decoupling akin to semantization. We compare BCPNN plasticity with the more commonly used spike-timing-dependent plasticity (STDP) learning rule in the same episodic memory task. Unlike BCPNN, STDP does not explain the decontextualization process. We further examine how selective plasticity modulation of isolated salient events may enhance preferential retention and resistance to semantization. Our model reproduces important features of episodicity on behavioral timescales under various biological constraints while also offering a novel neural and synaptic explanation for semantization, thereby casting new light on the interplay between episodic and semantic memory processes.


Introduction
Episodic and semantic memory were originally proposed as distinct systems that compete in retrieval (Tulving, 1972). More recent studies suggest, however, that this division is rather vague (McCloskey and Santee, 1981;Howard and Kahana, 2002;Renoult et al., 2019), as neural correlates of episodic and semantic retrieval overlap (Weidemann et al., 2019). Episodic memory traces are susceptible to transformation and loss of information (Tulving, 1972), and this loss of episodicity can be attributed to semantization, which typically takes the form of a decontextualization process (Viard et al., 2007;Habermas et al., 2013;Duff et al., 2020). Baddeley (1988) hypothesized that semantic memory might represent the accumulated residue of multiple learning episodes, consisting of information which has been semanticized and detached from the associated episodic contextual detail. Extensions of the classical Remember/Know (R/K) behavioral experiment demonstrated that item-context decoupling can occur rapidly (Opitz, 2010). In these experiments, items were presented either in a unique context, or across several contexts. Low context variability improved the recollection rate, whereas context overload led to decontextualization and "Know" type of responses, i.e., recognition of item-only information without any detail about episodic context (Opitz, 2010;Smith and Manzano, 2010;Smith and Handy, 2014). To the best of our knowledge, there have not been any computational hypotheses proposed to offer mechanistic insights into this item-context decoupling effect.
Several computational spiking neural network models of cortical associative memory have previously been developed and used to investigate mechanisms underlying working memory maintenance and recall (Lundqvist et al., 2010(Lundqvist et al., , 2011Herman et al., 2013). A similar model enhanced with a Bayesian-Hebbian learning rule (Bayesian Confidence Propagation Neural Network; BCPNN) representing synaptic and intrinsic plasticity was then used to study one-shot memory encoding (Fiebig and Lansner, 2017), and more recently, it was extended into a multinetwork cortical model to examine a novel "indexing theory" of working memory (Fiebig et al., 2020).
In the present study, relying on a similar spiking neural network model with identical modular architecture we propose and evaluate a Bayesian-Hebbian hypothesis about synaptic and network mechanisms underlying memory semantization and qualitatively match model output to available behavioral data. We show that associative binding between items and contexts becomes weaker when an item is presented across multiple contexts (high context variability). This gradual trace transformation relies on the nature of Bayesian learning, which normalizes and updates weights over estimated presynaptic (Bayesian-prior) as well as postsynaptic (Bayesian-posterior) spiking activity. We compare these findings with an analogous model that features the more well-known spike-timing-dependent plasticity (STDP) instead of the BCPNN learning rule, and demonstrate that no memory semantization effect can be reproduced, regardless of the degree of context variability. Notably, there have been earlier modeling attempts at semantization using STDP or other learning rules but this memory phenomenon has been interpreted differently involving slow memory consolidation (requiring sleep, repeated exposures, or systems consolidation) or extraction of semantic relations (a.k.a. prototype learning) among a group of episodic memories sharing statistical similarities (Deperrois et al., 2021;Remme et al., 2021). We argue that our hypothesis is more generic as it does not assume any statistical structure of the memory object representations. Finally, we also show how selective plasticity neuromodulation of one-shot learning (tentatively modeling effects of attention, emotional salience, and surprise on plasticity) may delay or prevent decontextualization.
In contrast to existing computational models of episodic memory (Norman and O'Reilly, 2003;Wixted, 2007;Greve et al., 2010), our model bridges behavioral outcomes with neural and synaptic mechanisms. It reproduces episodic memory phenomena on behavioral time scales under constrained network connectivity with plausible postsynaptic potentials, firing rates, and other biological parameters.

Neuron and synapse model
We use adaptive exponential integrate-and-fire point model neurons, which feature spike frequency adaptation, enriching neural dynamics and spike patterns, especially for the pyramidal cells (Brette and Gerstner, 2005). The neuron model is an effective model of cortical neuronal activity, reproducing a wide variety of electrophysiological properties, and offers a good phenomenological description of typical neural firing behavior, but it is limited in predicting the precise time course of the subthreshold membrane voltage during and after a spike or the underlying biophysical causes of electrical activity (Gerstner and Naud, 2009). We slightly modified it for compatibility with the BCPNN synapse model (Tully et al., 2014) by integrating an intrinsic excitability current.
Development of the membrane potential V m and the adaptation current I w is described by the following equations: Vm ÀV t Dt À I w 1I ext 1I syn (1) dI w dt ¼ ÀI w t Iw 1 bd ðt À t sp Þ: Equation 1 describes the dynamics of the membrane potential V m including an exponential voltage dependent activation term. A leak current is driven by the leak reversal potential E L through the conductance g L over the neural surface with a capacity C m . Additionally, V t is the spiking threshold, and D T shapes the spike slope factor. After spike generation, membrane potential is reset to V r . Spike emission upregulates the adaptation current by b, which recovers with time constant t I w (Table 1). To simplify the model, we have removed subthreshold adaptation, which is part of some AdEx models.
Besides a specific external input current I ext , model neurons receive synaptic currents I syn j from conductance based glutamatergic and GABAergic synapses. Glutamatergic synapses feature both AMPA/NMDA receptor gated channels with fast and slow conductance decay dynamic, respectively. Current contributions for synapses are described as follows: The glutamatergic synapses are also subject to synaptic depression and augmentation with a decay factor t D and t A , respectively (Table 1), following the Tsodyks-Markram formalism (Tsodyks and Markram, 1997). We have chosen those time-constants from the plausible range of computational fits made on the basis of electrophysiological recordings of cortical pyramidal cells (Wang et al., 2006). The utilization factor u represents the fraction of available resources used up by each transmitted spike (a proxy of synaptic release probability), whereas x tracks the fraction of resources that remain available because of transmitter depletion (synaptic depression):

Spike-based BCPNN plasticity
We implement synaptic plasticity of AMPA and NMDA connection components using the BCPNN learning rule (Lansner and Ekeberg, 1989;Wahlgren and Lansner, 2001;Tully et al., 2014). BCPNN is derived from Bayes rule, assuming a postsynaptic neuron employs some form of probabilistic inference to decide whether to emit a spike or not. Despite that it accounts for the basic Bayesian inference, it is considered more complex than the standard STDP learning rule (Caporale and Dan, 2008), and as such, it reproduces the main features of STDP plasticity.
The BCPNN synapse continuously updates three synaptic biophysically plausible local memory traces, P i , P j , and P ij , implemented as exponentially moving averages (EMAs) of preactivation, postactivation, and coactivation, from which the Bayesian bias and weights are calculated. EMAs prioritize recent patterns, so that newly learned patterns gradually replace old memories. Specifically, learning implements exponential filters, Z, and P, of spiking activity with a hierarchy of time constants, t z , and t p , respectively [the full BCPNN model implements additional eligibility E traces (Tully et al., 2014), which are not used here]. Because of their temporal integrative nature, they are referred to as synaptic (local memory) traces.
To begin with, BCPNN receives a binary sequence of presynaptic and postsynaptic spiking events (S i , S j ) to calculate the traces Z i and Z j : f max denotes the maximal neuronal spike rate, e is the lowest attainable probability estimate, t spike denotes the spike duration while t z i ¼ t z j are the presynaptic and postsynaptic time constants, respectively (t z ¼ t AMPA ¼ 5 ms for AMPA, and t z ¼ t NMDA ¼ 100 ms for NMDA components; Table 1). P traces are then estimated from the Z traces as follows: The parameter k adjusts the learning rate, reflecting the action of endogenous modulators of learning efficacy (i.e., activation of a D1R-like receptor). Setting k = 0 freezes the network's weights and biases, though in our simulations the learning rate remains constant (k = 1) during encoding (see Results, Semantization of episodic representations in the BCPNN model and Item-context interactions under STDP). However, we trigger a transient increase of plasticity in specific scenarios to model preferential retention of salient events (see Results, Preferential retention; Table 1).
Finally, P i , P j , and P ij are used to calculate intrinsic excitability b j and synaptic weights w ij with a scaling factor b gain and w syn gain respectively (Table 1): Spike-based STDP learning rule In our study, we examine the impact on semantization when the STDP learning rule replaces BCPNN associative connectivity in the same episodic memory task. Synapses under STDP are developed and modified by a repeated pairing of presynaptic and postsynaptic spiking activity, while their relative time window shapes the degree of modification (Ren et al., 2010). The amount of trace modification depends on the temporal difference (D t ) between the time point of the presynaptic action potential (t i ) and the occurrence of the postsynaptic spike (t j ) incorporating a corresponding transmission delay (t d ) from neuron i to neuron j: After processing Dt, STDP updates weights accordingly: Here, l corresponds to the learning rate, a reflects a possible asymmetry between the scale of potentiation and depression, t 6 control the width of the time window, while m 6 [ {0,1} allows to choose between different versions of STDP (i.e., additive, multiplicative; Morrison et al., 2008). Synapses are potentiated if the synaptic event precedes the postsynaptic spike and get depressed if the synaptic event follows the postsynaptic spike (Van Rossum et al., 2000).
Associative weights w ij are initialized to w 0 , and their maximum allowed values are constrained according to w max to ensure that synaptic weights are always positive and between [w 0 ; w max ] ( Table 2). The resulting associative weight distributions are generally comparable in strength to the BCPNN model weights, but to make them match, we adjust w max in conjunction with a reasonably small learning rate l . The maximum allowed weight (w max ) is a necessary standard parameter of the default STDP we use in NEST (see below, Code accessibility). To obtain a stable competitive synaptic modification, the integral of Dw ij must be negative (Song et al., 2000). To ensure this, we choose a = 1.2, which introduces an asymmetry between the scale of potentiation and depression along with a symmetric time window resulting in a ratio of at À =t 1 .1.0 (Ren et al., 2010). We set m 6 = 1 resulting in multiplicative STDP (in-between values lead to rules which have an intermediate dependence on the synaptic strength).

Two-network architecture and connectivity
The network model features two reciprocally connected networks, the so-called Item and Context networks. For simplicity, we assume that Item and Context networks are located at a substantial distance accounting for the reduced internetwork connection probabilities (Table 3). Each network follows a cortical architecture with modular structure compatible with previous spiking implementations of attractor memory networks (Lansner, 2009;Lundqvist et al., 2011;Tully et al., 2014Tully et al., , 2016Fiebig and Lansner, 2017;Chrysanthidis et al., 2019;Fiebig et al., 2020), and is best understood as a subsampled cortical layer 2/3 patch with nested hypercolumns (HCs) and minicolumns (MCs; Fig. 1A). Both networks span a regular-spaced grid of 12 HCs (Table 3), each with a diameter of 500 mm (Mountcastle, 1997). In our model, items are embedded in the Item network and context information in the Context network as internal well consolidated long-term memory representations (cell assemblies), supported via intranetwork weights derived using prior   MCs. We preload abstract long-term memories of item and context representations into the respective network, in the form of distributed cell assemblies with weights establishing corresponding attractors. Associative plastic connections bind items with contexts. The network features lateral inhibition via basket cells (purple and blue lines) resulting in a soft winner-take-all dynamics. Competition between attractor memories arises from this local feedback inhibition together with disynaptic inhibition between HCs. B, Weight distribution of plastic synapses targeting pyramidal cells. The attractor projection distribution is positive with a mean of 2.1, and the disynaptic inhibition is negative with a mean of À0.3 (we show the fast AMPA weight components here, but the simulation also includes slower NMDA weight components). C, Weight matrix between attractors and competing MCs across two sampled HCs. The matrix displays the mean of the weight distribution between a presynaptic (MC pre ) and postsynaptic MC (MC post ), within the same or different HC (black cross separates grid into blocks of HCs, only two of which are shown here). Recurrent attractor connections within the same HC are stronger (main diagonal, dark red) compared with attractor connections between HCs (off-diagonals, orange). Negative pyramidal-pyramidal weights (blue) between competing MCs amounts to disynaptic inhibition mediated by double bouquet cells.
Research Article: New Research MCs per HC (;100 MCs per HC in biological cortex). The high degree of recurrent connectivity within (Thomson et al., 2002;Yoshimura and Callaway, 2005) and between MCs links coactive MCs into larger cell assemblies (Stettler et al., 2002;Binzegger et al., 2009;Muir et al., 2011;Eyal et al., 2018). Long-range bidirectional internetwork connections (item-context bindings or associative connections) are plastic (shown in Fig. 1A only for MC1 in HC1 of the Context network), binding items, and contextual information (Ranganath, 2010). On average, recurrent connectivity establishes 100 active plastic synapses onto each pyramidal cell from other pyramidals with the same selectivity, because of a sparse internetwork connectivity (cp PPA ) and denser local connectivity (cp PP , cp PPL ; connection probability refers to the probability that there is a connection between a randomly selected pair of neurons from given populations; in Fig. 1A, connection probabilities are only shown for MC1 in HC1 of the Context network). The model yields biologically plausible EPSPs for connections within HCs (0.45 6 0.13 mV), measured at resting potential E L (Thomson et al., 2002). Densely recurrent nonspecific monosynaptic feedback inhibition mediated by fast spiking inhibitory cells (Kirkcaldie, 2012) implements a local winner-take-all structure (Binzegger et al., 2009) among the functional columns.
IPSPs have an amplitude of À1.160 mV (60.003) measured at À60 mV (Thomson et al., 2002). These bidirectional connections between basket and pyramidal cells within the local HCs are drawn with a 70% connection probability. Notably, double bouquet cells shown in Figure 1A are not explicitly simulated, but their effect is nonetheless expressed by the BCPNN rule. A recent study based on a similar single-network architecture (i.e., with the same modular organization, microcircuitry, conductancebased AdEx neuron model, cell count per MC and HC) demonstrated that learned mono-synaptic inhibition between competing attractors is functionally equivalent to the disynaptic inhibition mediated by double bouquet and basket cells (Chrysanthidis et al., 2019). Parameters characterizing other neural and synaptic properties including BCPNN can be found in Table 1. Figure 1B shows the weight distributions of embedded distributed cell assemblies, representing different memories stored in the Item and Context networks. Attractor projections can be further categorized into strong local recurrent connectivity within HCs, and slightly weaker long-range excitatory projections across HCs (Fig. 1C).

Axonal conduction delays
Conduction delays (t ij ) between a presynaptic neuron i and a postsynaptic neuron j are calculated based on their Euclidean distance, d, and a conduction velocity V (Eq. 11). Delays are randomly drawn from a normal distribution with a mean according to distance and conduction velocity, with a relative SD of 30% of the mean to account for individual arborization differences, and varying conduction speed as a result of axonal thickness and myelination. In addition, a minimal delay of 1.5 ms (t syn min ; Table 3) is added to reflect synaptic delays because of effects that are not explicitly modelled, e.g., diffusion of neurotransmitters over the synaptic cleft, dendritic branching, thickness of the cortical sheet and the spatial extent of columns (Thomson et al., 2002). Associative internetwork projections have a 10-fold faster conduction speed than those within each network, reflecting axonal myelination: Stimulation protocol Noise input to pyramidal cells and fast spiking inhibitory basket cells is a zero-mean noise, generated by two independent Poisson generators with opposing driving potentials. Pyramidal cells coding for specific items and contexts are stimulated with an additional specific excitation during encoding and cued recall (all parameters in Table 3). Item-context association encoding is preceded by a brief period of background noise excitation to avoid initialization transients.

Attractor activation detector
We detect and report cued recall of items or contexts by using an attractor activation detection algorithm based on EMAs of spiking activity. Pattern-wise EMAs are calculated using Equation 12, where the d function d denotes the spike events of a pattern-selective neural population of n pop = 30 pyramidal cells. The filter time constant t = 40 ms is much larger than the sampling time interval DT = 1 ms: Pattern activations are detected by a simple threshold (r th ) at about 10-fold the baseline activity with a small caveat: to avoid premature offset detection because of synchrony in fast spiking activity, we only count activations as terminated if they do not cross the threshold again in the next 40 ms. This method is highly robust because of the explosive dynamics of recurrent spiking activity for activated attractors in the network. Any attractor activation that crosses this threshold for at least 40 ms is considered a successful recall.

Semantization of episodic representations in the BCPNN model
An episodic memory task simulated in this work is inspired by a seminal memory effect shown in an experimental study by Opitz (2010). We deliberately abstract away some details of Opitz (2010) experimental design to provide a qualitative proof of principle with as few task assumptions as possible. This approach also offers a more generalized computational framework for studying the interplay of synaptic learning and its outcomes. In the same spirit, the systems architecture of our model is reduced to the Item and Context networks storing item and context information, respectively, as internal long-term memory representations ( Fig. 1; for details, see Two-network architecture and connectivity and Table 3). We stimulate some items in a single context and others in a few different contexts establishing multiple associations (Fig. 2). Stimulus duration during encoding is t stim = 250 ms with a T stim = 500 ms interstimulus interval, and a test phase occurs after a 1-s delay period, which contains brief t cue = 50-ms cues of previously learned items (Table 3). Figure 3A illustrates an item-context pair, established by an associative binding through plastic bidirectional BCPNN projections (dashed lines). Item and context attractors (solid red lines) are embedded in each network and remain fixed throughout the simulation, representing well-consolidated long-term memory. We show an exemplary spike raster of pyramidal neurons in HC1 of both the Item and Context networks reflecting a trial simulation (Fig. 3B). Herein, item-3 (blue) establishes a single association, while item-4 (yellow) is encoded in four different contexts (Figs. 2A, 3B). We find clear evidence for strong item-context decoupling. The yellow item-4 (but not the blue item-3) is successfully recognized when cued but without any corresponding accompanying activation in the Context network (Fig. 3B). Figure 3C demonstrates that this itemcontext decoupling effect holds true also for the multi trial average as the performance of contextual retrieval when items serve as cues deteriorates with a higher context overload. Successful item recognition without any contextual information retrieval accounts for a "Know" response, as opposed to "Remember" judgments, which are accompanied by context recall. In fact, episodic loss in our network implies that no context is recalled despite the item memory activation. To elucidate this observed progressive loss of episodicity with the higher context variability (Fig. 3C), we sample and analyze the learned weight distributions of itemcontext binding recorded after the association encoding period (Fig. 3D). The item-context weight distribution in the one-association case is significantly stronger than in the two-association, three-association, or four-association case (p , 0.001, Mann-Whitney, N = 2000). This progressive weakening of weights leads to significantly lower EPSPs for the associative projections (p , 0.05 for one vs two associations; p , 0.001 for two vs three and three vs four associations, Mann-Whitney, N = 300; Fig. 3D, see inset). To measure EPSPs, we stimulate individually all the neurons in HC1 of an item which forms one, two, three, or four   Fig. 2A). Each item is drawn with a unique color, while contexts inherit their coactivated item's color in the raster (i.e., the yellow pattern in the Item network is repeated over four different contexts, forming four separate associations marked with the same color). The testing phase occurs 1 s after the encoding. Brief 50-ms cues of already studied items trigger their activation. Following item activation, we detect evoked attractor activation in the Context network. C, Average cued recall performance in the Context network (20 trials). The bar diagram reveals progressive loss of episodic context information (i.e., semantization) over the number of context associations made by individual cued items (compare Fig. 2A). D, Distribution of plastic connection weights between the Item and Context networks (NMDA component shown here). Weights are noticeably weaker for items which participate in multiple associations. The distributions of synaptic weights exhibit a broader range for the items with multiple context associations, as the sample size is larger. The inset displays the distribution of EPSPs for the binding between Item and Context networks. The EPSP distributions follow the trend of the associative weights. The amplitudes (,1 mV) are lower for higher context variability. E, The distribution of intrinsic excitability currents of pyramidal cells coding for specific context representations. The intrinsic excitability features similar distributions because each context is activated exactly once, regardless of whether the associated item forms multiple associations or not. F, Average cued recall performance in the Item network (20 trials). Decontextualization over the number of associations is also observed when we briefly cue episodic contexts instead (compare Fig. 2B). G, Distribution of strength of plastic connections from the contexts to their associated items. Analogously to D, synapses weaken once an item is encoded in another context. H, Intrinsic plasticity distribution of cells in the Item network. Intrinsic excitability distributions are higher for pyramidal cells coding for repeatedly activated items; ***p , 0.001 (Mann-Whitney, N = 20 in C, F). Error bars in C, F represent SDs of Bernoulli distributions. Distributions of one, two, three, and four associations in D, G, H show significant statistical difference (p , 0.001, Mann-Whitney, N = 2000). associations and record the postsynaptic potential onto their associated context neurons. EPSPs are in general below 1 mV (Thomson et al., 2002;Song et al., 2005), measured at resting potential E L (Table 1), after item-context association encoding phase. Therefore, we attribute the loss of episodicity to a statistically significant weakening of the associative weight distributions with the increasing number of associated contexts. The associative weight distributions shown here refer to the NMDA component, while the weight distributions of the faster AMPA receptor connections display a similar trend (data not shown). The gradual trace modification we observe is a product of Bayesian learning, which normalizes and updates weights over estimated presynaptic (prior) as well as postsynaptic (posterior) spiking activity (see below, BCPNN and STDP learning rule in a microcircuit model for details).
Our simulation results are in line with related behavioral studies (Opitz, 2010;Smith and Manzano, 2010;Smith and Handy, 2014), which also reported item-context decoupling as the items were presented across multiple contexts. In particular, Opitz (2010) concluded that repetition of an item across different contexts (i.e., high context variability) leads to item-context decoupling, which is in agreement with our study. Furthermore, Smith and Manzano (2010) demonstrated in an episodic context variability task configuration, that recall deteriorates with context overload (number of words per context). Mean recall drops from ;0.65 (one word per context) to 0.50 (three words per context), reaching ;0.33 in the most overloaded scenario (fifteen words per context).
In Figure 3E, we show the distribution of intrinsic excitability over units representing different contexts. Pyramidal neurons in the Context network have a similar intrinsic excitability, regardless of their selectivity because all the various contexts are encoded exactly once.
Next, analogously to the previous analysis, we show that item-context decoupling emerges also when we briefly cue contexts rather than items during recall testing (Fig. 2B). In agreement with experimental data (Smith and Manzano, 2010;Smith and Handy, 2014), we obtain evidence of semantization as items learned across several discrete contexts are hardly retrieved when one of their associated contexts serves as a cue (Fig. 3F). We further sample and present the underlying associative weight distribution, between the Context and the Item networks (Fig. 3G). The distributions again reflect the semantization effect in a significant weakening of the corresponding weights. In other words, an assembly of pyramidal neurons representing items encoded across multiple contexts receives weaker projections from the Context network. At about four or more associations, the itemcontext binding becomes so weak that it fails to deliver sufficient excitatory current to trigger associated representations in the Item network. At the same time, intrinsic excitability of item neurons increases with the number of associated contexts corresponding to how much these neurons were active during the encoding phase (

Item-context interactions under STDP
In this section, we contrast the results obtained with the BCPNN synaptic learning rule with those deriving from the more commonly used STDP learning rule in the same episodic memory task ( Fig. 2; Spike-based STDP learning rule). The modular network architecture as well as neural properties and embedded memory patterns remain identical, but associative projections between networks are now implemented using a standard STDP synaptic learning rule (Morrison et al., 2008). The parameters of the STDP model are summarized in Table 2. Figure 4A shows an exemplary spike raster of pyramidal cells in HC1 of both the Item and the Context networks, based on the first variant of the episodic memory task described in Figure 2A. As earlier, items are encoded in a single or in multiple different contexts and they are briefly cued later during recall. A successful item activation may lead to a corresponding activation of its associated information in the Context network. We detect these activations as before (see Materials and Methods, Attractor activation detector) and report the cued recall score over the number of associations (Fig. 4B).
Unlike the BCPNN network, we observe no evidence of semantization for high context variability. Instead, recollection is noticeably enhanced with an increase in the number of associations, which is in fact the opposite of what would be needed to explain item-context decoupling. STDP generates similarly strong associative binding regardless of context variability (Fig. 4C). The enhanced recollection in high context variability cases stems from the multiplicative effect of synaptic augmentation in the Tsodyks-Makram model on the Hebbian attractor weights. Items stimulated multiple times (e.g., four times) have a higher likelihood of being encoded near the end of the task, leading to more remaining augmentation during testing, thus, effectively boosting cued recall (Fig. 5A). This recency effect diminishes after removing synaptic augmentation from the model as attractor weights in the Item network have comparable distributions leading to similar cued recall performance regardless of context variability (Fig. 5B,C). As far as the context-cued variant of the task is concerned (Fig. 2B), there are also no signs of item-context decoupling for high context variability (Fig.  4D). The associative projections between Context and Item networks again have distributions with comparable means over context variability (Fig. 4E). Overall, decontextualization is not evident in either variant of the episodic memory task under the STDP learning rule.

BCPNN and STDP learning rules in a microcircuit model
To better elucidate the emergent synaptic changes of the BCPNN and STDP model, we also apply these learning rules in a highly reduced microcircuit of spiking neurons. To this end, we now track the synaptic weight changes continuously. The neural and synaptic parameters (and most importantly all the plasticity parameters) used for the highly reduced BCPNN and STDP model are identical to the ones used for the large scale BCPNN and STDP model, respectively (see Tables 1, 2).
First, we apply the BCPNN learning rule to the microcircuit model. We consider two separate item neurons (ID = 1 and 2), which form two or three associations with context neurons (ID = 3, 4, or 5, 6, 7), respectively (Fig.  6A). We display the synaptic strength development of the synapse between item neuron-1 and context neuron-3 (two associations, green), as well as the synapse between item neuron-2 and context neuron-5 (three associations, red) over the course of training these associations via targeted stimulation. BCPNN synapses get strengthened when the item-context pairs are simultaneously active and weaken when the item in question is activated with another context. Therefore, synapses of the item neuron that is encoded in three different contexts converge on weaker weights (Fig. 6A, 12 s), than those of the item neuron with two associated contexts. Weight modifications in the microcircuit model reflect the synaptic alterations observed in the large-scale network. BCPNN weights are shaped by traces of activation and coactivation (Eqs. 7, 8; Materials and Methods), which also get updated during the activation of an item within another context. For example, the item neuron-1 and context neuron-3 are not . Network model where associative projections are implemented using standard STDP synaptic plasticity. A, Spike raster of pyramidal neurons in HC1 of both the Item and Context networks. B, Average item-cued recall performance in the Context network (20 trials). Episodic context retrieval is preserved even for high context variability (as opposed to BCPNN; compare Fig. 3C). C, Distribution of NMDA receptor mediated synaptic weights between the item and context neural assemblies following associative binding. The distributions of item-context weights have comparable means at ;0.065 nS regardless of how many context associations a given item forms. Bins merely display a higher count for the four-association case as the total count of associative weights is more extensive compared with items with fewer associations. D, Average cued recall performance in the Item network when episodic contexts are cued (20 trials). E, Distribution of NMDA component weights between associated context and item assemblies; ***p , 0.001 (Mann-Whitney, N = 20 in B, D). Error bars in B, D represent SDs of Bernoulli distributions. stimulated together between 6 and 8 s, but neuron-1 and context neuron-4 are. Thus, the P traces of the item activation (P i ) increase, while the ones linked to context-3 (P j ) decay with a time constant of 15 s (Table 1). Since the item and context neuron (ID = 1, 3) are not stimulated together, their coactivation traces (P ij ) decay between 6 and 8 s. Overall, this leads to a weakening of the weight and hence to a gradual decoupling (Eq. 8; Materials and Methods).
In the same manner, we keep track of weight change in a microcircuit with the STDP learning rule (Fig. 6B). Unlike the microcircuit with BCPNN presented in Figure 6A, the STDP weights corresponding to the associations made by both item neurons converge to similar values, although they are associated with different number of contexts. As before, the synapse between an item neuron and an associated context neuron strengthens when this pair is simultaneously active, but remains stable when the item neuron is encoded in another context. For instance, the synapse between item neuron-2 and context neuron-5 strengthens when this pair is encoded (0-2 s), yet remains unaffected when item neuron-2 is activated in another context (i.e., context neuron-6, 4-6 s). This synaptic behavior explains the observed differences between the BCPNN and STDP largescale model.

Preferential retention
Several studies propose that one-shot salient events promote learning, and that these memories can be retained on multiple time scales ranging from seconds to years (Frankland et al., 2004;Petrican et al., 2010;Gruber et al., 2016;Panoz-Brown et al., 2016;Eichenbaum, 2017;Sun et al., 2018). Hypothetical mechanisms behind these effects are dopamine release and activation of DR1 like receptors, resulting in synapse-specific enhancement (Otmakhova and Lisman, 1996;Kuo et al., 2008), and systems consolidation (McClelland et al., 1995; Fiebig and  Neural and synaptic parameters correspond to those in the scaled model. In both cases, two item neurons (ID = 1,2) are trained to form two or three associations, respectively (dashed connections are simulated but their weight development is not shown here). During training, neurons are stimulated to fire at 20 Hz for 2 s. We display the developing synaptic weight between specific item-context pairs (ID = 1 and 3 in the 2-association scenario) and (ID = 2 and 5 in the 3-association scenario), and compare the converged weight values between the two-association and three-association case under both learning rules, following a final readout spike at 11 s. . Overall, salient or reward driven events may be encoded more strongly as the result of a transient plasticity modulation. Recall from long-term memory is often viewed as a competitive process in which a memory retrieval does not depend only on its own synaptic strength but also on the strength of other components (Shiffrin, 1970). In view of this, we study the effects of plasticity modulation on encoding specific items within particular contexts, with the aim of investigating the role of enhanced learning for semantization in our model.
Using the same network and episodic memory task as before ( Fig. 2A), we modulate plasticity during the encoding of item-1 (red) in context-E via k = k boost (Eq. 7; Materials and Methods; Table 1). This results in an increased cued recall probability for the item associated with three episodic contexts relative to the unmodulated control (Fig. 7A, Normal vs Biased scenario, three associations). Episodic retrieval improves from 0.6 (Normal; Fig.  7A, left) to 0.8 (Biased, modulated plasticity; Fig. 7A, right) when item-1 is cued, which now performs more similarly to item with just two associated contexts. We further analyze and compare the recall of each context when its associated item-1 is cued (Fig. 7B, three associations). The control scenario (Normal; Fig. 7B, left) without transient plasticity modulation shows that the three contexts (ID = A, E, and J) are all recalled with similar probabilities. In contrast, encoding a specific pair with enhanced learning (upregulated k = k boost ) yields higher recall for the corresponding context. In particular, the plasticity enhancement during associative encoding of the context-E (with item-1) results in an increased recall score to 0.8 (0.25 control), while the other associated contexts, ID = A and J, are suppressed (Fig. 7B), primarily because of soft winner-take-all competition between contexts (Fig. 1A).
We attribute these changes to the stronger weights because of enhanced learning (Fig. 7C, dark red distribution, g ). Weights between unmodulated item-context pairs (item-1 and context-A,-J) show mostly unaltered weight distributions (a,b , light red), while the biased associative weight distribution between item-1 and context-E is now comparable to the weight distribution of the one-association case. Performance does not exactly match that case though because of some remaining competition among the three contexts. Overall, these results demonstrate how a single salient episode may strengthen memory traces and thus impart resistance to semantization (Rodríguez et al., 2016).

Discussion
The primary objective of this work was to explore the interaction between synaptic plasticity and context variability in the semantization process. To cast new light on the episodic-semantic interplay, we built a cortical memory model of two spiking neural networks that feature the same modular architecture. The networks are coupled with plastic associative connections, which collectively represent distributed cortical episodic memory. Our results suggest that some forms of plasticity offer a synaptic explanation for the cognitive phenomenon of semantization, thus bridging scales and linking network connectivity and dynamics with behavior. We use a spiking neuronal network model combined with BCPNN, which allows us to directly compare it with a standard Hebbian STDP learning rule. In particular, we demonstrated that with Bayesian-Hebbian (BCPNN) synaptic plasticity, but not with standard Hebbian STDP, the model can reproduce traces of semantization as a result of learning. Notably, this was achieved with biologically constrained network connectivity, postsynaptic potential amplitudes, and firing rates compatible with mesoscale recordings from cortex and earlier models. Nevertheless, our hypothesis of the episodic-semantic interplay at a neural level requires further experimental study of the synaptic strength dynamics in particular. As mentioned, quantitative data on cortical synaptic plasticity is still quite limited, and while STDP has been shown to offer explanation for Figure 7. Plasticity modulation of a specific item-context pair enhances recollection and counteracts semantization. A, Context recall performance. One of the pairs (context-E, item-1) presented in the episodic memory task (compare Fig. 2A) is subjected to enhanced plasticity during encoding, resulting in the boosted recall rate (3 associations, Normal vs Biased, 20 trial average). B, Individual context retrieval contribution in the overall recall (3 associations). Retrieval is similar among the three contexts since plasticity modulation is balanced (left: Normal, k = k normal ; compare Table 1). However, when context-E is encoded with enhanced learning (with item-1), its recall increases significantly (right: Biased, k = k boost ; compare Table  1) some associative memory phenomena (Pokorny et al., 2020), any specific plasticity rule is insufficiently validated experimentally. Yet our results with BCPNN offer a possible explanation and testable behavioral predictions. The spiking version of this plasticity rule has repeatedly been shown to be compatible with detailed, biologically constrained network activity and structure. Importantly, our simulations clearly demonstrate how cognitive phenomena such as semantization could be produced and thus explained by microscopic plasticity processes. In particular, BCPNN solves the issue of decontextualization by its information-theoretical principle, not by being hand-crafted to do so. Like any Bayesian estimator, BCPNN trades-off synaptic strength (weights) for increased intrinsic excitability (bias) in highly active neurons, thus decreasing synaptic strength of neurons that are highly active outside of a specific spiking correlation. Unlike many other conceivable learning rules that might achieve this effect, BCPNN is working only on locally available information, and is thus also biologically plausible.
Our study conforms to related behavioral experiments reporting that high context variability or context overload leads to item-context decoupling (Opitz, 2010;Smith and Manzano, 2010;Smith and Handy, 2014). These studies suggest that context-specific memory traces transform into semantic representations while contextual information is progressively lost. Traces of item memory representations remain intact but fail to retrieve their associated context. Semantization is typically described as a decontextualization process that occurs over time. However, several experiments, including this study, proposed that exposures of stimuli in different additional contexts (rather than time itself) is the key mechanism advancing semantization (Opitz, 2010;Smith and Manzano, 2010;Smith and Handy, 2014). In fact, simple language vocabulary learning implies that learners encode words in several different contexts, which leads to semantization and a definition-like meaning of the studied word (Beheydt, 1987;Bolger et al., 2008). Although our network model is limited to a simple item-context decoupling scenario, the proposed plasticity mechanistic explanation for the observed item-context decoupling effect may be generalized to support semantization in more complex scenarios in which other mechanisms may synergistically interact and contribute to decontextualization. Admittedly, our hypothesis does not exclude other seemingly coexisting phenomena and mechanisms supporting memory retrieval that may facilitate semantization over time, e.g., reconsolidation or systems consolidation because of sleep or aging (Friedrich et al., 2015). Further, our model does not feature any higherorder mechanisms allowing a neutral stimulus (lacking prior pairing) to evoke the same contextual memory response as a conditioned stimulus does despite their prior pairing. In other words, each stimulus has to be independently coupled with its context(s).
We also demonstrated (Results, Preferential retention) how a transient plasticity modulation, reflecting known isolation effects, may preserve episodicity, staving off decontextualization. Semantization may also be overcome by accumulating additional evidence regarding an episode. In our simulations, we typically used single context cues to retrieve an item during cued recall (e.g., item-4 forms four associations, but only one of its associated contexts was cued, compare Figs. 2B and 3F). However, an interesting question is whether providing multiple context cues that share the same target item boosts its recall (Fig. 8A). Figure 8B shows the result where we sequentially stimulated all the four different contexts in the fourassociation case. The 4-fold contextual information considerably increases the likelihood of retrieval of a nearly fully semanticized item (compare Figs. 3F and 8B, four Figure 8. Average cued recall performance in the Item network after sequentially cueing all the contexts that are associated with the item that forms four associations. A, Spike raster of pyramidal neurons in HC1 of both the Item and Context networks. The cue paradigm during test for the one-association, two-association, and three-association case remains identical to the control case (compare Fig. 2B). However, in particular for the four-association case, we sequentially cue all the four available contexts that share the same target item. B, Average cued recall performance in the Item network (20 trials). The bar diagram reveals progressive loss of item information over the number of context associations, but not for the four-association case at which all the available contexts were cued during test. Thus, providing more evidence via different sources boosts retrieval (;95%) recovering a nearly decontextualized item (compare Fig. 3F, four associations, single cue, 25% accuracy score); ***p , 0.001 N = 20). Error bars represent SDs of Bernoulli distributions. associations). These results are relatively intuitive yet novel from a modeling point of view and in line with behavioral studies reporting enhanced cued recall with multiple cues compared with a single one (Rubin and Wallace, 1989;Broadbent et al., 2020;Pearson and Wilbiks, 2021).
To our knowledge, there is no other spiking computational model of comparable detail that captures the semantization of episodic memory explored here, while simultaneously offering a neurobiological explanation of this phenomenon. Unlike other dual-process episodic memory models, which require repeated stimulus exposures to support recognition (Norman and O'Reilly, 2003), our model is able to successfully recall events learned in "one shot" (a distinctive hallmark of episodicity). We note that the attractor-based theory proposed in this study does not exclude the possibility of a dual-process explanation for recollection and familiarity (Yonelinas, 2002;Yonelinas et al., 2010).

Related models of familiarity and recollection
Perceptual or abstract single-trace dual-process computational models based on signal detection theory explain episodic retrieval but the potential loss of contextual information is only implied as it does not have its own independent representation (Wixted, 2007;Greve et al., 2010). These computational models often aim to explain traditional R/K behavioral studies. As discussed earlier, participants in such studies are instructed to give a "Know" response if the stimulus presented in the test phase is known or familiar without any contextual detail about its previous occurrence. Conversely, "Remember" judgments are to be provided if the stimulus is recognized along with some recollection of specific contextual information pertaining to the study episode. This results in a strict criterion for recollection, as it is possible for a subject to successfully recall an item but fail to retrieve the source information (Ryals et al., 2013). Numerous studies suggest that recollection contaminates "Know" reports because recalling source information sensibly assumes prior item recognition (Wais et al., 2008;Johnson et al., 2009). Mandler (1979Mandler ( , 1980 and Atkinson and Juola (1973) treat familiarity as an activation of preexisting memory representations. Our results are compatible with this notion because our model proposes to treat item-only activations as "Know" judgments, while those accompanied by the activation of context representations best correspond to a "Remember" judgment. Item activation is a faster process and precedes context retrieval (Yonelinas and Jacoby, 1994), and our model reflects this finding by necessity, as item activations are causal to context retrieval.
We assume that familiarity recognition is simply characterized by lack of contextual information, yet the distinction we make between the Context and Item networks is arbitrary. Memory patterns stored in the Context network are referred to as contexts and those in the Item network as items. From the perspective of the network's architecture, items and contexts have representations of the same nature, nonoverlapping sparse distributed patterns. While sparse internetwork connectivity is sufficient for our model's function, both networks may just as well be part of the same cortical brain area. The actual physical separation of the two networks (which incurs connection delays commensurate with the axonal conduction speed) is motivated by our assumption that items and contexts are not necessarily represented within the same network. A more specific scenario might assume that items and contexts share part of the same local network. In principle, our model should be capable of replicating similar results in that case.

Biological plausibility and parameter sensitivity
We investigate and explain behavior and macroscale system dynamics with respect to neural processes, biological parameters of network connectivity, and electrophysiological evidence. Our model consequently builds on a broad range of biological constraints such as intrinsic neuronal parameters, cortical laminar cell densities, plausible delay distributions, and network connectivity. The model reproduces plausible postsynaptic potentials (EPSPs, IPSPs) and abides by estimates of connection densities (i.e., in the associative pathways and projections within each patch), axonal conductance speeds, typically accepted synaptic time constants for the various receptor types (AMPA, NMDA, and GABA), with commonly used neural and synaptic plasticity time constants (i.e., adaptation, depression).
The model synthesizes a number of functionally relevant processes, embedding different components to model composite dynamics, hence, it is beyond this study to perform a detailed sensitivity analysis for every parameter. Instead, we provide insightful observations for previously unexplored parameters that may critically affect semantization. Importantly, a related modular cortical model already investigated sensitivity to important short-term plasticity parameters (Fiebig and Lansner, 2017). After extensive testing we conclude that the model is generally robust to a broad range of parameter changes and its performance only gradually degrades in terms of the effect size. We expect even lower sensitivity to parameter variations in a network approaching biological scales. Further, it is worth reporting that the model's function is preserved across a wide range of sizes of a cortical column as long as the number of pyramidal cells is not excessively low. The same holds for the population of the inhibitory basket cells provided that the rough total inhibitory synaptic current is maintained by controlling feedback synaptic strength (g BP ; Table 3).
The P trace decay time constant, t p , of the BCPNN model is critical for the learning dynamics modelled in this study because it controls the speed of learning in associative connections and the resulting weight amplitude. High values of t p imply long-lasting but weaker memory traces and therefore lead to slower, more inertial learning (more resistance to change and encoding new information as well as forgetting) with overall lower weights and hence weaker binding. Varying t p by 630% does not change the main outcome, i.e., episodicity still deteriorates with a higher context variability. At the same time, as mentioned, slower weight development results in weaker associative binding and overall lower recall (and vice versa for faster learning). To compensate for this loss of episodicity, an additional increase in the unspecific input is usually sufficient to trigger comparable recall rates. Alternatively, the recurrent excitatory gain can be amplified to complete noisy inputs toward discrete embedded attractors. Unspecific background input during recall plays a critical role as well. In general, we use a low background noise input into the two coupled networks. However, for the enhanced noise by 140% the model operates in a free recall regime with spontaneously reactivating memories without any external cues.
As we explained in the investigation of the reduced microcircuit model, semantization is an inherent property of BCPNN-driven weight dynamics, derived from Bayesian logic. However, countervailing forces in local microcircuits contribute to the generation and maintenance of our associated memories, Bayesian weight development drives semantization while intrinsic plasticity counteracts it. In consequence, it is possible to lessen the relative impact of this synaptic weight-dependent effect by making intrinsic plasticity more prominent: frequently activated items (in varying contexts) become more excitable because of memory recency effect. Conversely, we can maximize the semantization effect by making the bias current weaker, though fully removing it (b gain = 0) is hard to justify from a biological perspective. By manipulating the strength of intrinsic plasticity (b gain ) to diminish signs of decoupling as described above, the capacity, i.e., the number of retrievable item-context associations, can increase beyond three associations (compare Fig. 3). Other key factors that can enhance model capacity for itemcontext associations (resistance to semantization over many established episodic associations) are larger network size, higher associative binding connection probability (e.g., increase in cp PPA from 2% to 4%; Table 3), and elevated background unspecific noise during the cue-response association period. Strengthening associative binding by upregulating w syn gain can also enhance the model's capacity for item-context associations (Table 1). Still, there is an upper limit to w syn gain as extreme values can lead to implausible EPSPs.
This study also demonstrates how a selective transient increase of plasticity can counteract semantization. The plasticity of the model can be modulated via the parameter k (Eq. 7; Materials and Methods). Typically, k is set to 1 (k = k normal ; Table 1), whereas we double plasticity (k = k boost ; Table 1), when modeling salient episodic encoding. We notice that by selectively tripling or quadrupling plasticity (relative to baseline) during encoding of a specific pair whose item component forms many other associations, the source recall improves progressively (data shown only for k = k boost in Results, Preferential retention).
Finally, in Results, BCPNN and STDP learning rule in a microcircuit model, we compare STDP and BCPNN plasticity in a highly reduced model. We bind items with contexts to form different number of associations and keep track of the weight development per time step. STDP plasticity generates same magnitude item-context binding regardless of how many associations an item forms. A detailed parameter analysis for every critical synaptic parameter (630%) did not yield any behaviorally significant changes to the converged weights.

Semantization over longer time scales
Source recall is likely supported by multiple independent, parallel, interacting neural structures and processes since various parts of the medial temporal lobes, prefrontal cortex and parts of the parietal cortex all contribute to episodic memory retrieval including information about both where and when an event occurred (Gilboa, 2004;Diana et al., 2007;Watrous et al., 2013). A related classic idea on semantization is the view that it is in fact an emergent outcome of systems consolidation. Sleep-dependent consolidation in particular has been linked to advancing semantization of memories and the extraction of gist information (Payne et al., 2009;Friedrich et al., 2015).
Models of long-term consolidation suggest that retrieval of richly contextualized memories become more generic over time. Without excluding this possibility, we note that this is not always the case, as highly salient memories often retain contextual information (which our model speaks to). Instead, our model argues for a much more immediate neural and synaptic contribution to semantization that does not require slow multiarea systems level processes that have yet to be specified in sufficient detail to be tested in neural simulations. It has previously been shown, however, that an abstract simulation network of networks with broader distributions of learning time constants can consolidate memories across several orders of magnitude in time, using the same Bayesian-Hebbian learning rule as used here (Fiebig and Lansner, 2014). That model included representations for prefrontal cortex, hippocampus, and wider neocortex, implementing an extended complementary learning systems theory (McClelland et al., 1995), which is itself an advancement of systems consolidation (Squire and Alvarez, 1995). We consequently expect that the principled mechanism of semantization explored here can be scaled along the temporal axis to account for lifelong memory, provided that the plasticity involved is itself Bayesian-Hebbian. Our model does not advance any specific anatomic argument as to the location of the respective networks (Yonelinas, 2002;Diana et al., 2007). The model purposefully relies on a generic cortical architecture focused on a class of synaptic plasticity mechanisms which may well serve as a substrate of a wider system across brain areas and time.
In conclusion, we have presented a computational mesoscopic spiking network model to examine the interplay between episodic and semantic memory with the grand objective to explain mechanistically the semantization of episodic traces. Compared with other models of episodic memory, which are typically abstract, our model, built on various biological constraints (i.e., plausible postsynaptic potentials, firing rates, connection densities, synaptic delays, etc.) accounting for neural processes and synaptic mechanisms, emphasizes the role of synaptic plasticity in semantization. Hence it bridges micro and mesoscale mechanisms with macroscale behavior and dynamics. In contrast to standard Hebbian learning, our Bayesian version of Hebbian learning readily reproduced prominent traces of semantization.