Generalization Through the Recurrent Interaction of Episodic Memories

Relationship to Exemplar-Based Models of Memory

The softmax function used to determine the activity of units in the conjunctive layer grounds REMERGE in classical frameworks of similarity-based generalization and exemplar-based models of memory (Nosofsky, 1984; see below and the Appendix). Exemplar models have been influential in accounting for patterns of behavior observed in a range of domains from episodic memory tasks (e.g., recognition tasks) to categorization (Hintzman, 1986; Medin & Schaffer, 1978; Nosofsky, 1984; Shiffrin & Steyvers, 1997). These models assume that each experience tends to result in the creation of an individual memory trace, which from a geometric perspective can be viewed as represented by a point in a high dimensional space. As such, performance in a variety of settings, for example the assignment of a given stimulus to one of two categories in a categorization task, is thought to be supported by the retrieval of the most similar previously experienced exemplars, rather than any more abstract (e.g., prototype) representations.

The REMERGE model, based on localist coding of individual episodes in the conjunctive layer, can be considered a close cousin of exemplar-based models, though incorporating an additional principle of recurrence. In particular, the hedged softmax activation function used in the conjunctive layer produces activations of conjunctive units that are directly analogous to the normalized probe-exemplar similarity values calculated in an influential member of the exemplar model class, the GCM model (Nosofsky, 1984; see the Appendix for overview of principles of the GCM model and its relationship to REMERGE). As such, the activity of a unit on the network's conjunctive layer at each timestep can be viewed as reflecting the similarity of the corresponding premise pair (or stored exemplar) to the current pattern of activity on the feature layer. Processing in REMERGE, therefore, is closely related to that carried out by exemplar models: In both settings, generalization results from the parallel activation of multiple conjunctive traces determined by their similarity to the current input (the nature of which is regulated by the temperature or sensitivity parameter). In simulations presented in the Appendix, we demonstrate that REMERGE retains the capacity of exemplar-based models to perform categorization and recognition memory—the latter simulation provides a proof-of-principle demonstration that recurrency within the hippocampal circuit is broadly compatible with its well-established role in recognition memory.

We highlight one salient difference between our model and classical exemplar models: Specifically, REMERGE includes a recurrent mechanism that enables similarity-based inference and generalization to be performed not only on externally presented sensory inputs (i.e., test stimuli displayed on the screen) but also on feature layer inputs reconstructed by the network. This mechanism, which we refer to as recurrent similarity computation, is critical to the ability of REMERGE to perform successfully in settings like the B–D trial in the transitive inference task (see below)—by allowing the inherent similarity of two related experiences (e.g., B–C, D–E in the transitive inference task) to be captured, even when pairwise similarities in input-space are uninformative (and in this case equate to zero). While the utility of recurrency in “cleaning up” the output of the model during cued recall was noted in a previous exemplar model (MINERVA 2; e.g., the category name retrieved in response to presentation of the prototype stimulus), only limited explorations of this mechanism were performed, apparently due to limitations in available computational resources (Hintzman, 1986).

Relationship to Interactive Activation and Competition (IAC) Networks

While our use of the softmax activation function was motivated by a desire to emphasize the close relationship between our model and similarity based mechanisms of generalization, it is important to highlight parallels that exist between the REMERGE model and a class of IAC networks (McClelland, 1981; McClelland & Rumelhart, 1981; Nystrom & McClelland, 1992). As previously noted, IAC models have been used to capture a wide range of phenomena involving generalization, ranging from contextual effects in pronounceable non-words as well as words to conceptual learning tasks (McClelland, 1981; McClelland & Rumelhart, 1981; Nystrom & McClelland, 1992), and these models were our starting place in developing the framework presented here. In the first application of the IAC network architecture to memory (McClelland, 1981), the model was considered to have stored the attributes of a set of fictitious characters belonging to two rival gangs (Jets and Sharks). As in the current model, feature and conjunctive layers were comprised of localist units, connected by bidirectional excitatory connections. In this case, feature units denoted the properties of individuals (e.g., name, occupation, marital status, age, education), while units in the conjunctive layer represented different individuals (e.g., Lance). A key property of the network's mechanism of generalization is well illustrated by the following example: First, the relevant weights specifying the occupation of one particular individual (e.g., Lance: burglar) were temporarily deleted. Next, the network's ability to “fill in” Lance's appropriate occupation was tested by activating the Lance name unit on the feature layer. Critically, the network exhibited the phenomenon of generalization—in this case activating the burglar feature unit—through recurrent activation. Activating Lance's name unit activated the conjunctive-layer unit for Lance, and this in turn activated Lance's features (other than the burglar feature). This in turn led to the activation of conjunctive units coding for individuals sharing several attributes with Lance (i.e., Jim, John, George), and these conjunctive units in their turn sent activation to the burglar unit, given that all of these individuals share the burglar occupation property. Such generalization would not have occurred in standard versions of exemplar models (e.g., Nosofsky, 1984), since exemplar activations are determined solely by the direct similarity between the cues provided externally (in this case, the name, Lance) and items stored in memory. Since only one item in memory has the name Lance, no other items would be activated by the probe. It is the activation of other properties of Lance, as a result of recurrence, that provides the basis for generalization in this case, since it is only through the activation of these other properties that the activation of items sharing these properties can occur.

In summary, this example illustrates how recurrency in the IAC model (McClelland, 1981) allows an important extension of the generalization capacity of an exemplar-based mechanism. REMERGE builds on this feature of the IAC model, incorporating it into the overall framework of exemplar models of learning and memory. Overall, therefore, the REMERGE model can be considered to support a capacity for generalization through a process of recurrent similarity computation, based on the essential operating principles of both exemplar-based models and interactive activation networks.

Connections of the REMERGE Model With Explicit Probabilistic Inference

There are also strong connections between the REMERGE model and Bayesian inference. We note, first of all, that the logistic and softmax functions used in REMERGE match very standard Bayesian computations, if connection weights and bias terms are set to be equal to the logarithms of well-known Bayesian quantities (related to the prior and the likelihood, respectively; see Hinton & Sejnowski, 1983; McClelland, 1998; Mirman et al., in press). We note further that a strong connection has recently been noted between exemplar models and approximate Bayesian inference (Shi, Griffiths, Feldman, & Sanborn, 2010); here, a key point is that the e^net term in our equations corresponds to the similarity term in exemplar models, and this in turn can be viewed as corresponding to the likelihood that the current input to an exemplar model would have been observed as a presentation of one of the exemplars stored in memory. To the extent that REMERGE corresponds to a classical exemplar model, then, it also corresponds to an approximate Bayesian computation.

There are two particular ways in which REMERGE differs from a standard Bayesian computation, however. The first difference lies in the fact that REMERGE treats the connection weights as quantities that change over time as knowledge of an item becomes stronger. Clearly, the weights in this case will not capture true likelihoods. They may, however, approximate varying degrees of knowledge of those true likelihoods (McClelland & Chappell, 1998). The second is in the use of recurrent activation, which, as stated above, allows REMERGE to extend its inferential capabilities beyond those of standard exemplar models. We believe there is a natural Bayesian interpretation of this extension, but we do not develop it here, since our primarily goal is to consider whether our model can account for patterns in data. Further explorations are needed to consider these issues more fully.

Transitive Inference Paradigm

We first enquire whether a hippocampal system operating over pattern separated representations can support rapid generalization in the transitive inference task through an interaction of conjunctive units coding for related training episodes (e.g., BC, CD) mediated by recurrency. The term transitive inference refers to the ability to infer a relationship between stimuli (e.g., B, D) that have never been experienced together, based on previous learning of overlapping premise pairs (i.e., BC, CD). In the transitive inference task, subjects are rewarded for choosing one member of each premise pair over another (e.g., A+ B−, B+ C−, etc.). Typically, four or five pairs are used altogether, extending to D+ E−, or E+ F−, respectively. After successfully learning the premise pairs to a predefined criterion, subjects are tested on their ability to select the correct stimulus in an inference pair (i.e., B+ D−). Typically, transitivity performance improves as a function of the amount and nature (i.e., interleaved vs. blocked) of training given on premise pairs, and also as a function of the delay interposed between the end of premise pair training and inferential test (Ellenbogen et al., 2007; Moses et al., 2006; Ryan et al., 2009).

It is agreed that the hippocampus makes an important contribution to transitivity (cf. premise pair performance) across species (Eichenbaum, 2004; Zeithamova et al., 2012). While rats with hippocampal dysfunction and control subjects required a comparable number of trials to reach criterion on premise pairs, lesioned animals performed at chance levels on the critical B–D inference pair (Dusek & Eichenbaum, 1997). However, the neural mechanisms underlying the role of the hippocampus in transitivity remain unclear and subject to considerable debate. Two general classes of theories have been proposed to account for transitivity performance, those that fall under the umbrella of “associative linking” according to our terminology (Eichenbaum, 2004; Howard et al., 2005; McGonigle & Chalmers, 1977; O'Reilly & Rudy, 2001; Wu & Levy, 2001) and those that involve a form of value transfer (Frank, Rudy, & O'Reilly, 2003; von Fersen, Wynne, Delius, & Staddon, 1991). Associative linking theories, which can be applied to a range of inferential tasks, argue that generalization arises from interactions between adjacent episodes (e.g., BC, CD), typically through overlap in the relevant neuronal codes (see General Discussion for a description of the temporal context model; Howard et al., 2005) (Eichenbaum, 2004; McGonigle & Chalmers, 1977; O'Reilly & Rudy, 2001; Wu & Levy, 2001). Value transfer theories (Frank et al., 2003; von Fersen et al., 1991), in contrast, argue that transitivity performance results primarily from a difference in the reinforcement value of individual stimuli (e.g., B vs. D). The idea here is that the value of each stimulus is determined not only by its individual reinforcement history, which is equivalent for middle items (e.g., B, D in a five-item series) but additionally by the context in which it is rewarded. In this way, items (e.g., B) nearer the always rewarded “A” end anchor tend to accrue higher values than those nearer (e.g., D) to the never rewarded “E” item (in a four-pair series). As a result, a subject may correctly choose B over D in a four-pair series simply because it has a higher reward value, rather than as a consequence of any more elaborate associative linking mechanism.

Our aim here is not to summarize what is a vast literature on the subject of transitive inferences (for reviews, see Breslow, 1981; Delius & Siemann, 1998; Moses et al., 2006) or to provide an account that encompasses the full range of findings observed within the broad range of transitive inference studies. Indeed, it is widely agreed that multiple neural mechanisms contribute in parallel to performance in the transitive inference and, indeed, other mnemonic tasks, depending on the exact training conditions imposed and the species concerned (e.g., Moses et al., 2006; Zeithamova et al., 2012). Rather, we limit our focus to that of proposing a mechanistic basis for the contribution of an associative linking process to inference. We do this since associative linking mechanisms have broad relevance outside the specific domain of transitivity (unlike value transfer mechanisms), both in the setting of the paired associative inference and acquired equivalence tasks considered subsequently, as well as in other inferential paradigms and semantic learning more generally. Further, the notion of associative linking naturally invokes a consideration of the role of the hippocampus to inferential behavior, given its capacity to rapidly set up memory representations for arbitrary conjunctions of individual items, of the sort encountered during premise trials.

Simulation of Transitive Infererence

Simulation of a Basic Capacity for Transitive Inference

We first illustrate the basic capacity of our recurrent network to exhibit the phenomenon of transitivity, with the aim of laying bare the mechanism of generalization—namely, the process of recurrent similarity computation. Notably, the characteristics of the sigmoidal neural transfer function (i.e., softmax rule) used to determine the activity of units in the conjunctive layer allow the functioning of our network to be interpreted within well-established frameworks of similarity computation and exemplar models (see the Appendix for an overview of the principles of exemplar models of memory and their relationship to REMERGE).

In the interests of clarity, we examine the timecourse of network activity during a B–D trial in more detail (see Figure 2, top panel). Activation of the B and D units on the feature layer spreads to the four relevant conjunctive units (AB, BC, CD, DE) over the first few cycles. In effect, the initial equivalence of the activity of these four units can be thought of as denoting that the current featural input (i.e., B, D) is equally similar to all of these previously learnt premise pairs. In a probabilistic sense, activity of a given conjunctive unit can be viewed as representing a posterior probability that the current pattern of feature activity (i.e., B, D unit activity) was generated through the distorted reoccurance of the relevant premise pair (e.g., BC). Over successive network cycles, the network reconstructs a new pattern of feature layer activity based on this initial similarity computation performed by the conjunctive layer, leading to the activity of the C unit dominating (over other units except those receiving direct external input), since it receives convergent input from both BC and CD units. Subsequently, this new pattern of feature layer activity (i.e., primarily over the B, C, and D units) causes the BC and CD units to dominate over the DE and AB units in the conjunctive layer: in effect, because the new reconstructed feature layer input is more similar to the BC and CD premise pairs (cf. the DE pair). Finally, the conjoint activation of the BC and CD conjunctive units drives the B unit to win over the D unit in the response layer.

The network's performance during a B–E inference trial (see Figure 2, middle panel) and a B–C premise trial (see Figure 2, bottom panel) is also illustrated. In a B–E trial, as in a B–D trial, the network's capacity for transitivity is mediated by a process of recurrent similarity computation, with the activity of the BC and DE units rising over the AB and EF units, despite the fact that the stimuli presented in the trial (i.e., B–E) share one feature in common with all four of these conjunctive units. Further, it can also be appreciated that the operation of recurrency results in the final activity of the CD unit, which has no features in common with the stimuli presented, being greater than that of the AB and EF units. It is worth noting, however, that in a B–C premise trial (see Figure 2, bottom panel), the presence of a direct match between the stimulus combination and previous experience results in dominant activity of the BC unit, with relatively little activation of other conjunctive units.

Generalization, therefore, in the model can be considered an emergent phenomenon, which arises a process of recurrent similarity computation, rather than through stored representations of the linear hierarchical task structure. As noted previously, traditional models of similarity based generalization (Hintzman, 1986; Medin & Schaffer, 1978; Nosofsky, 1984; Shepard, 1987), typically lacking a capacity for recurrency, are unable to perform inference in the transitivity and related tasks. This is the case because recurrency allows similarity-based computations to operate over not only externally presented sensory inputs (i.e., B and D in a B–D probe trial) but also over feature layer activity patterns reconstructed by the network, a function critical to the phenomenon of transitivity.

Here, we restrict the focus of the current treatment to the generalization capacities of the REMERGE model, given the clear links between processing in this network and a rich literature on similarity/exemplar models. Nevertheless, it is worth noting that an IAC network (using standard activation function, a resting level of zero, and parameters as detailed in the PDP Handbook: http://www.stanford.edu/group/pdplab/pdphandbook/) is able to reproduce a basic capacity for transitivity (e.g., in a B–D trial), with the pattern of network activity observed closely mirroring that exhibited by the REMERGE model (see Figure 3). While a formal analysis of their exact relationship is likely to be complex, it would seem that generalization in an IAC network, at least in the current setting, may involve a form of recurrent similarity computation broadly analogous to that performed by the REMERGE model.

Simulating the Emergence of Inferential Capacity Through the Strengthening of Premise Pair Weights

Having demonstrated the core mechanism by which REMERGE performs transitivity, we next asked how the emergence of a capacity for inference might be simulated in the network. Our intuition was that the development of transitive responding might arise as a natural consequence of the strengthening of individual premise pair memories, implemented as an increase in the strength of the relevant weighted connections in the network. To explore this hypothesis, we adopted an idealized scheme of learning in the network (see above), whereby the first experience with an item (e.g., A–B premise pair in the transitive inference task) creates a weak representation, modeled by a localist unit with very weak initial incoming and outgoing connection weights, which is incrementally strengthened by subsequent experiences during continued training. As such, we examined the relationship between the network's performance on premise pairs and inference pairs across a range of different weight strengths.

Here, we see clearly that in the model, relatively weak connection weights can support some level of premise performance, without yet supporting generalization (see Figure 4, upper panel): As weights get stronger, and premise performance becomes more robust, a tendency to produce transitive inference responses emerges. Importantly, the network's failure to exhibit transitivity at lower weight strengths results from its failure (see Figure 5) to develop a graded pattern of activation over the relevant units in the feature and conjunctive layers, and in particular to exhibit greater activation of the C unit, compared to the E unit, in the feature layer during a B–D inference trial. Further, it is also evident that there is a decrement in the network's generalization performance relative to premise performance at each weight strength. The magnitude of this “lag generalization” effect is notably dependent on the setting of the temperature parameter: Lower temperature values (see Figure 4, lower panel) result in premise performance reaching near-asymptotic values before generalization performance rises significantly above chance levels, whereas higher values (e.g., Figure 4, upper panel) result in the parallel rise of premise and generalization performance, with generalization performance lagging only slightly behind premise performance.

Transitive Inference: Empirical Data and Simulation: Ryan et al. (2009)

Having demonstrated the basic capacity of REMERGE to produce transitivity, and having established the core mechanism (i.e., recurrent similarity computation) by which it achieves this, we next turned our attention to relevant empirical data. The basic phenomenon of transitivity, and its disruption by hippocampal system damage, has been observed by numerous studies (Dusek & Eichenbaum, 1997; Greene et al., 2001; Heckers et al., 2004; McGonigle & Chalmers, 1977; Moses et al., 2006; Ryan et al., 2009; C. Smith & Squire, 2005; Van Elzakker et al., 2003). Few studies (Moses et al., 2006; Ryan et al., 2009; also see Greene et al., 2001), however, have examined how the capacity for transitivity develops over the course of learning by charting the relationship between premise pair performance and inference pair performance. Given the relative paucity of suitable data, our aim in these simulations, and indeed throughout this article, was to ask whether the network can reproduce the essential features of the behavioral data. Accordingly, we do not attempt to achieve a close fit to specific aspects of individual experiments through an extended search of the parameter space, but rather we consider these studies as illustrative examples representative of the extant literature. We focus on one example from the recent human literature, where the emergence of a capacity for transitivity was assessed at several timepoints over the course of training (Ryan et al., 2009).

We consider the performance of 20 young healthy volunteers who participated in the experiment by Ryan et al. (2009). The stimulus set consisted of six abstract visual pictures (A–F), yielding five premise pairs with items presented on either side of the screen, with location randomized between trials. On each premise pair trial, subjects were required to choose an item, receiving correct/incorrect feedback depending on their response. Premise pair training was conducted in five stages: In Stage 1, premise pairs were presented 10 times in consecutive order (i.e., 10 × AB, 10 × BC … 10 × EF). In Stage 2, premise pairs were presented five times in consecutive order (i.e., 5 × AB, 5 × BC … 5 × EF). Stage 3 was divided into three blocks, with premise pairs being presented three times in each block in consecutive order. During Stage 4, each premise pair was presented once in consecutive order for nine trials. In the final stage of training, premise pairs were interleaved and presented in pseudorandom order, for a total of 18 repetitions of each pair. A testing block followed each training stage in which both premise pairs (e.g., AB) and inference pairs (e.g., BD) were presented without corrective feedback. Following the final stage of the experiment, subjects' awareness of the linear hierarchy of stimuli was assessed.

As shown in Figure 6 (upper panel), premise and inferential performance increased in parallel across the five stages of training in Ryan et al.'s (2009) study. No significant difference was observed between performance on close (i.e., B–D, C–E) and distant inference pairs in this experiment. The results of our simulation, capturing the same general pattern as seen in the data, are also illustrated in the figure (lower panel). As noted previously, the overall tendency of participants' generalization performance to lag slightly behind premise performance is reflected in the mild lag generalization effect exhibited by the model where the temperature parameter is set to a relatively high value (i.e., 0.45; see Figure 4, upper panel, for illustration of relationship between weight strength and network performance at these parameter settings).

Summary of Model Simulations

1. We demonstrate a basic capacity of the network to perform transitive inference, based on a principle of big-loop recurrence within the hippocampal system, which mediates the interaction of multiple related conjunctive units resulting in graded patterns of network activity.

2. Importantly, the network achieves inference using localist conjunctive units, designed to mirror a fundamental principle of the coding scheme employed by the hippocampus, namely pattern separation.

3. Generalization in REMERGE results from a process of recurrent similarity computation. Critically, the activity of conjunctive units can be viewed to reflect their similarity to not only externally presented sensory inputs but also inputs reconstructed by the network on the feature layer. This enables the network to exploit indirect relationships between items presented in different training experiences. In this way, recurrence can be considered to expand the explanatory power of exemplar-based models to include inferential paradigms such as the transitivity task.

4. While we focus on the REMERGE model, whose network characteristics reveal close parallels with existing exemplar based models of similarity computation, we also demonstrate that the phenomenon of transitivity is also a property of IAC networks, with processing proceeding along similar lines. We suggest, therefore, that a wider class of recurrent networks related to REMERGE is likely to display generalization capacities of the form described.

5. We show that the emergence of a capacity for transitivity can be simulated by changing a single parameter, namely the strength of weighted connections coding for premise pairs. As such, progressive training may facilitate the development of a capacity for inference (e.g., Ryan et al., 2009), merely by strengthening individual memory traces for the premise pairs themselves, rather than necessitating the discovery of new representations of the (linear) structure of the training set.

6. A characteristic feature of the model is that generalization performance tends to lag behind premise performance, as the strength of weights coding for premise pair memories is increased. Within the model, the approximately parallel rise of premise and generalization performance observed in Ryan et al.'s (2009) study is consistent with a relatively low setting of the temperature parameter, which gives rise to a similar pattern in the simulation (see Figure 4, upper panel, and Figure 6).

Paired Associate Inference Task

We first examined the capacity of the model to perform inference in a paradigm originally developed by Bunsey and Eichenbaum (1996) and used in a related form in a recent human imaging experiment (Zeithamova & Preston, 2010; Zeithamova et al., 2012; also see Preston et al., 2004). In the original paradigm (Bunsey & Eichenbaum, 1996), rats were first trained on premise pairs in two stages: In Stage 1, subjects were presented with cue odors (A or X) and were required to learn which of two choice odors (B or Y) was rewarded (i.e., A: B+ Y−: X: B− Y+). In the second stage, the choice odors were presented as cues, and subjects were required to choose appropriately among two new choice odors (i.e., B: C+ Z−, Y: C− Z+). Having been trained to criterion on these premise pairs, subjects then entered the generalization testing phase. In this phase, subjects were presented with cue odors from the first phase (i.e., A or X) and were required to choose between choice odors from the second training phase (i.e., C or Z) in the absence of corrective feedback. Successful generalization was indexed by the choice of C over Z in the context of cue A (i.e., A: C+ Z−) and Z over C in the context of cue X (i.e., X: C− Z+). Critically, lesions of the hippocampus produced a specific impairment in a capacity for generalization, while leaving premise pair performance relatively unaffected (i.e., no significant difference between control subjects and lesioned animals in number of trials to reach criterion on premise trials; Bunsey & Eichenbaum, 1996).

The human fMRI experiment (n = 23 subs) by Zeithamova and Preston (2010) was conducted along similar lines to these, but with some notable exceptions. Subjects intentionally learnt premise associations between overlapping pairs of objects (i.e., AB, BC, XY, YZ) in a single exposure and were not required to choose a stimulus during the study period. The entire stimulus set consisted of 144 overlapping object pairs. As in Bunsey and Eichenbaum (1996), a premise pair test trial required subjects to choose, for example, Object B over another object (e.g., Y), in the context of Object A (termed an AB premise pair trial). In generalization test trials, subjects were required to choose C over Z in the context of A (termed an AC generalization or transfer trial), and vice versa in the context of X. Successful generalization in the PAI task, therefore, involves appreciating the indirect relationship between items (e.g., A and C) presented in overlapping experiences at study (i.e., AB and BC).

Paired Associate Inference Task: Summary of Key Empirical Findings and Conclusions (Zeithamova & Preston, 2010)

1. Subjects performed well (average 82%) during the test phase of the experiment on premise pairs (i.e., AB trials; see Figure 7, top panel).

2. As a group, performance on generalization test trials (e.g., AC trials) was significantly above chance (average 64%; see Figure 7, top panel). Interestingly, however, generalization performance was found to vary greatly between individual subjects (range = 47%–90%). A binomial test was used to divide subjects into those that performed significantly above chance levels (“good” group, n = 12; mean performance = 73%) and those that did not (“poor” group, n = 11; mean performance = 54%). While performance on premise pairs was well above chance in both groups (good group: 85%, poor group: 77%), performance was significantly higher in the good group.

3. A key aim of the current study was to reveal the neural mechanisms underlying the ability of subjects to generalize in AC transfer trials. While the authors report findings relating to both stages of the experiment (i.e., study, test), here we focus on an observation of particular interest that relates to neural activity during the encoding (i.e., study) phase of the experiment (see Figure 8). Zeithamova and Preston (2010) performed a trial-by-trial subsequent memory analysis (Wagner et al., 1998), sorting object picture sets (e.g., AB, BC = 1 set) according to whether subjects successfully generalized during the test phase or not (i.e., AC correct vs. AC incorrect). This procedure enabled them to ask in which regions does neural activity during encoding (i.e., AB, BC trials) correlate with later successful generalization. Interestingly, they observed that activity within the hippocampus was greater during BC trials, but not during AB trials, when subjects later chose correctly (cf. incorrectly) on AC generalization trials.

The authors suggested that successful generalization was supported by integrated representations, or stored generalizations, which directly encoded the indirect relationship between items presented in related study experiences (i.e., a novel A–C association). Accordingly, the difference in generalization performance between good and poor subjects in their experiment was proposed to indicate a qualitative difference in the nature of neural representations accessible (i.e., integrated vs. premise, respectively). Based on the neural data, they further argued that the hippocampus was critical to the formation of such novel A–C associations, a process effected during BC encoding trials, and was indexed by the specific correlation observed between neural activity during BC trials and subsequent transfer success.

The study by Zeithamova and Preston (2010) provides a rich behavioral and neural data set indexing the emergence of successful generalization performance. As in the transitive inference task described above, we first asked whether a basic capacity for generalization can emerge from a recurrent network operating over localist codes for individual premise pairs. Next, we considered whether the behavioral performance of the good and poor generalizer groups in the current experiment could be accounted for by a strengthening of premise pair weights within the network, rather than necessarily implying a qualitative difference in the nature of neural representations in the hippocampus (i.e., integrated AC representations in the good group only). Finally, we consider the idea that the observed correlation between hippocampal activity during BC encoding trials and subsequent transfer ability may simply reflect the tendency for stronger memory traces for the individual premise pairs (e.g., AB) to favor both pattern completion (i.e., of A) during BC encoding trials and efficient generalization. As such, we aimed to link the functioning of the network to premise pair performance, generalization performance, and neural signals in the hippocampus, thereby providing a mechanistic account of the genesis of inferential behavior.

Model Specifics

As illustrated in Figure 9, the model was set up along the lines described for the transitive inference task covered previously and was designed to capture learning and generalization relating to overlapping sets of paired associates (e.g., AB, BC; XY, YZ; etc.). For the purposes of the simulation, two sets of paired associates were employed: The feature layer comprised six units denoting individual objects (A, B, C, X, Y, Z). The conjunctive layer comprised four units coding for premise pairs learnt during training (i.e., AB, BC, XY, YZ). Bidirectional excitatory connections were present between feature layer and conjunctive layer. Unidirectional excitatory connections were present between the conjunctive and response layer, the latter denoting the four objects that could be chosen during a given test trial (i.e., B, C, Y, Z). Note that feedforward inhibitory connections from the conjunctive layer to the response layer were absent in this simulation (cf. the transitive inference simulation) to mirror the specific experimental design used by Zeithamova and Preston (2010)—that is, the incorrect alternative choice in a given trial (e.g., X in a premise trial involving Objects A and B) had never previously been associated with negative feedback. In other respects (e.g., activation rules, free parameters, etc.), the model used here was directly analogous to that described above.

Prior to testing, the network was considered to have learned the premise pairs: This was coded through positive weights in the relevant connections between feature layer and conjunctive layer (e.g., A and B units in the feature layer to the AB unit in the conjunctive layer; B and C units in the feature layer to the BC unit in the conjunctive layer) and between the conjunctive layer and response layer (e.g., AB unit in conjunctive layer to B unit in response layer; BC unit in conjunctive layer to C unit in response layer).

Testing conditions were designed to mimic the conditions of the actual experiment: As such, premise pair test trials involved presenting external input to, for instance, the A, B, and Y units on the feature layer. Generalization test trials consisted of presenting external input to, for example, the A, C, and Z feature units. Importantly, therefore, the model, and the subjects in the experiment itself, had to select between objects of equal familiarity (e.g., C and Z in a generalization trial). As previously, the behavioral performance of the network was indexed by the Luce choice ratios, determined by the final activities of units in the response layer.

As before, we examined a highly simplified version of learning in the model: The strength of the excitatory weights between feature and conjunctive layers, and conjunctive and response layers, was fixed during each network simulation but varied across simulations—that is, testing was performed with the network in recall mode. The strengthening of weights in the network coding premise pairs was designed to explore whether variation in the degree of learning could lead to differences in generalization and, therefore, simulate the observed dissociation between good and poor generalizer groups.

Simulation Results: Behavioral Data

We first examined the basic capacity of the network to behave appropriately during both premise trials (e.g., A: B+ Y−) and generalization trials (A: C+ Z−). As before, the generalization capacity of the network is mediated through graded activity patterns within the network, reflecting the interaction of conjunctive units for related premise episodes (data not shown; parameters: τ = 0.2; C = 1; weight strength = 1). In particular, the development of patterns of activity in the network can also be viewed within a framework of recurrent similarity computation based on both externally presented input and patterns of activity reconstructed on the feature layer through pattern completion. Specifically, the externally driven activation of the A, C, and Z feature units initially results in equivalent activity of the AB, BC, and YZ conjunctive units, all of which are equally similar to the current featural input. In the next phase, this pattern of conjunctive activity reconstructs a new pattern of feature layer activity that comprises greater activity of the B unit (cf. the Y unit), driven by convergent activity from the AB and BC units. In turn, this feature layer activity pattern drives the AB and BC units to out-compete the YZ unit and, therefore, cause the C unit to win over the Z unit in the response layer.

Having established the basic capacity of the recurrent network to perform inference in this setting, we next examined its ability to account for the observed differences in behavioral performance between poor and good groups (Zeithamova & Preston, 2010), through a change in the strength of weights coding individual premise pairs (e.g., AB, BC). Empirical and simulated data are shown in Figure 7, showing that the model can indeed capture the profile of behavioral performance of poor and good groups through a difference in weight strength (i.e., 1.47 and 1.52, for poor and good groups, respectively).

Neural Data (Zeithamova & Preston, 2010)

The previous simulation suggests that the behavioral difference found at the group level in the experiment by Zeithamova and Preston (2010) may be accounted for through a quantitative change in the strength of premise pair memories. We next asked whether the subsequent memory effect reported, specifically that trial-by-trial variations in hippocampal activity during BC (but not AB) trials was correlated with subsequent performance in AC generalization test trials, could be explained in a similar fashion. Our hypothesis, therefore, was simply that better learning (i.e., greater weight strengths) of individual premise pairs (e.g., AB, BC) might underlie both successful behavioral generalization and the neural effects reported. Specifically, we reasoned that an incorrect choice on a particular AC test trial may reflect weaker memories of the relevant individual premise pairs (AB, AC). In contrast, correct AC performance may arise from stronger (i.e., weight strength = 1.52) premise memories. To test this hypothesis, we wished to examine overall levels of activity in the conjunctive layer during simulated AB and BC study trials, at weight strengths known to support either good or poor generalization ability (based on the simulation results discussed above).

Simulation Results

We therefore examined overall levels of network activity during simulated AB and BC encoding trials, at these different weight strengths (1.47 vs. 1.52). It is worth noting several points with regards to this simulation: In a simulated AB trial, we assume that the feedforward connections from the A and B feature units to the AB conjunctive unit are present (at a weight strength of either 1.47 or 1.52), but the feedback connections are absent. Given the BC pair had not yet been experienced during an AB trial, the relevant connections were absent in the network. Likewise, at the time of a BC trial, we assume that both feedforward and feedback connections link the A and B units on the feature layer to the AB conjunctive unit; however, only feedforward connections link the B and C units on the feature layer to the BC conjunctive unit. We relate overall activity (i.e., summed) across the conjunctive layer to the hippocampal fMRI effects observed, in line with our view of the relationship between the layers of our model and the anatomy of the hippocampus.

Simulated data are illustrated in Figure 8 (lower panel). Critically, the difference in overall activity levels in the conjunctive layer during encoding trials, as a function of subsequent performance on AC transfer trials (i.e., AC correct vs. incorrect), is considerably greater during BC trials compared to AB trials. These results show that it is possible to account for the observed correlation between hippocampal activity during a BC trial and subsequent performance in AC generalization test trials simply through the modulation of the strength of individual premise pairs (e.g., AB, BC). Further inspection of the simulated data suggests that increased hippocampal activity in BC trials associated with successful AC transfer is driven by activity of the AB unit (see Figure 8, lower panel)—in line with the view of Zeithamova and Preston (2010) that the process of pattern completion (i.e., the reactivation of the previous related AB experience) makes a significant contribution to the increased hippocampal activity observed under these circumstances. It should be noted, however, that Zeithamova and Preston further argued that the simultaneous activation of the AB and BC units may naturally lead to the formation of an integrated memory trace. While we would not wish to exclude this possibility—and consider the possible contribution of stored generalizations to inference in the General Discussion—our simulations suggest that a parsimonious mechanism based on the differential strength of memories for the studied experiences may be sufficient to account for the empirical data.

Acquired Equivalence Task

We next considered the phenomenon of acquired equivalence (AE), a classic paradigm involving generalization, which bears similarities to both the transitive inference and PAI tasks. In a typical AE experiment, illustrated by a recent experiment by Shohamy and Wagner (2008), an individual learns that two stimuli (e.g., faces F1 and F2) are functionally equivalent, in that they share an association with the same outcome (e.g., scene S1). One of the stimuli (F1) is also associated during training with another scene (S2), though the other is not (F2). Critically, during inference trials in the test phase of the experiment, subjects tend to choose scene S2, over another scene (e.g., S4), when confronted with face F2, based on the functional equivalence of F1 and F2 developed during training. Importantly, therefore, the successful generalization in the AE task, as in the PAI task, necessarily involves the exploitation of indirect relationships between items presented in different training experiences (e.g., faces F1 and F2).

Empirical Data: Shohamy and Wagner (2008)

This fMRI experiment followed similar lines as described above: There were, however, 12 pairs of antecedent stimuli (i.e., F1, F2/F3, F4, etc.), rather than two, as is typical in AE paradigms. During the training phase, subjects were presented with a face (F1) and were required to choose between two different scenes (S1, S3), receiving feedback for a correct choice (S1). Importantly, the incorrect scene for one face (S3) was the correct choice for a different face on another trial (e.g., F3), preventing simple stimulus-response learning. Face-scene combinations (F1–S1) were interleaved during training, with each encountered on eight occasions, allowing subjects to reach a high level of performance (∼90%). During a test phase that followed training, subjects' ability to choose appropriately on premise trials and inference trials was assessed, without the provision of feedback.

Acquired Equivalence Task: Summary of Key Empirical Findings and Conclusions (Shohamy & Wagner, 2008)

1. As a group, subjects successfully generalized during the test phase of the experiment (∼75% across the whole group), thereby showing the acquired equivalence phenomenon, tending to choose S2, for instance, over scene S4 when confronted with face F2 during inference trials (see Figure 10, upper left panel).

2. As in the experiment by Zeithamova and Preston (2010), subjects varied in terms of their generalization performance (from 38% to >90%), even though premise performance was similar (∼90%). Critically, a significant amount of variance in between-subjects generalization performance was accounted for by differences in the time course of activation in the hippocampus (and midbrain) during learning (see Figure 10). Specifically, in the “good” group, which was based on a median split of generalization performance (mean performance > 90%), a ramping up of activity in a hippocampal-midbrain circuit was observed from early to late training (see Figure 10, lower right panel), the degree of which correlated with subsequent generalization performance (see Figure 10, lower left panel). In contrast, no such increase in hippocampal activity was observed in the “poor” group (mean generalization performance ∼60%).

The authors suggest that successful generalization in the good group, though not the poor group, may be supported by integrated representations or stored generalizations reflecting the relationship between sets of individual training experiences. Based on the neural data, they further suggest that hippocampal pattern completion during premise pair (e.g., F2–S1) trials is critical to this process, operating through the retrieval of related training episodes (e.g., F1–S1) and the subsequent creation of integrated representations (e.g., linking the three elements F1, F2, and S1: F1–F2–S1).

As in previous simulations, we asked whether the differing behavioral and neural profiles observed in the good and poor generalizer subject groups could be parsimoniously accounted for by a strengthening of premise pair weights within the network, rather than necessarily implying a qualitative difference in the nature of neural representations in the hippocampus (i.e., integrated representations in the good group only).

Specifics of Model Architecture

The architecture of the model was similar to that used to simulate the transitive inference and PAI paradigms described previously.

Shohamy and Wagner (2008) employed 12 pairs of face stimuli (F1, F2). We, however, used only two pairs of face stimuli (i.e., F1, F2/F3, F4) and their respective scenes (S1, S2/S3, S4) to model the relevant effects. Of note, similar results were obtained when the full stimulus set comprising 24 faces and 24 scenes was used. In the model, eight processing units in the layer correspond to stimuli F1–F4 and S1–S4; six units in conjunctive layer correspond to the six trained face-scene pairings (i.e., F1S1, F1S2, F2S1, F3S3, F3S4, F4S3); four units in the response layer correspond to the network's choice (i.e., S1–S4). The following connections were present in the model: bidirectional excitatory connections between feature and conjunctive layers; unidirectional excitatory connections between conjunctive and response layers. Of note, as in the PAI simulations, there were no inhibitory feedforward connections from conjunctive to response layer, in keeping with the structure of the paradigm used by Shohamy and Wagner (i.e., the incorrect choice in a generalization trial [e.g., S4 in an F2S2S4 trial] had never previously been associated with negative feedback).

Prior to testing, the network was considered to have learned the premise pairs: for example, denoting that scene S1 should be chosen over S3 when face F1 is present. This was coded through positive weights in the relevant connections between feature layer and conjunctive layer (e.g., F1 and S1 in the feature layer to the F1S1 unit in the conjunctive layer) and between the conjunctive layer and response layer (e.g., F1S1 unit in conjunctive layer to S1 unit in response layer). Generalization test trials, for example, consisted of presenting external input to, for example, the F2 S2 and S4 visual units. As in the PAI task, therefore, the model, and the subjects in the experiment itself, had to select between scenes of equal familiarity (e.g., S2 and S4).

As before, we explored whether the strengthening of weights in the network coding premise pairs could lead to differences in generalization and, therefore, simulate the observed behavioral and neural dissociations between good and poor generalizer groups. Simulated data at both behavioral and neural levels are shown in Figure 11. The figure shows that the network is able to account for the behavioral differences between poor and good generalizer groups through a difference in the strength of connection weights coding for premise pairs (see Figure 11, top right panel). Further, the network is able to reproduce the key feature of the neural differences observed, specifically the greater rise in hippocampal activity during F2–S1 trials in the good group over the course of training (see Figure 11, top right panel). However, it should be noted that the simulated poor group also shows a rise in conjunctive activity through training, a feature that differs from the profile observed in the empirical data.

It can also be seen (see Figure 11, bottom right panel) that the higher conjunctive activity levels observed in the simulated good group (cf. poor group) during F2–S1 trials in the late stage of training is primarily due to greater activity of the F2–S1 unit, which directly matches the input presented during this trial type. Indeed, in a previous simulation, we have shown that when the stimulus combination presented during a trial directly matches a previous experience, the activity of the relevant conjunctive unit (i.e., F2–S1) dominates over all others (transitivity simulation: see Figure 2, lower panel). Our simulations, therefore, suggest that the increase in hippocampal BOLD signal observed during F2–S1 trials may not be driven primarily by the activation of related items (i.e., F1–S2, F1–S1) through pattern completion, as was the case in the simulation of the paired associate inference task (see Figure 8, lower panel). Our simulations suggest the alternative possibility that the observed increase in hippocampal activity during F2–S1 trials may be driven by activation of the directly matching conjunctive unit (i.e., F2–S1).

Summary of Paired Associate Inference and Acquired Equivalence Model Simulations and Conclusions

1. We considered the mechanism of generalization in the PAI and AE paradigms, both tasks in which the elemental values of stimuli are uninformative (cf. the transitivity task), demanding an appreciation of the higher order relationships between items presented in a set of related experiences.

2. The simulations presented provide additional support for the hypothesis that a recurrent hippocampal system, based on pattern separated neural codes and a process of recurrent similarity computation, is compatible with efficient generalization in these settings. As such, our model challenges the need for actual overlap at the representational level (Eichenbaum et al., 1999; Howard et al., 2005), a mechanism that has been argued to limit the efficiency of the hippocampus as a neural system supporting episodic memory (e.g., McClelland et al., 1995; Treves & Rolls, 1992).

3. The REMERGE model also provides a mechanistic account of why a proportion of subjects performing the PAI and AE tasks are able to generalize successfully in transfer trials, and others are not, relying simply on a difference in the strength of weights coding individual premise pairs.

4. We also suggest that our perspective provides a parsimonious mechanism by which to explain the observed relationship between trial-by-trial neural activity in the hippocampus during encoding, and subsequent generalization performance in the PAI and AE tasks. Specifically we suggest that strong premise weights lead to greater activation of either conjunctive units coding for overlapping premise experiences (i.e., AB unit during a BC trial in the PAI task) or conjunctive units that directly match the stimulus input (i.e., F2–S1 unit during a F2–S1 trial in the AE task).

5. Our model, therefore, provides a simple account of the genesis and emergence of generalization behavior, raising the question of when, and how, integrated representations (e.g., novel A–C associations in the PAI task) contribute to performance. That said, we consider in the General Discussion section how recurrency in the hippocampal system, operating during learning and over offline delays, may naturally give rise to stored generalizations of this nature.

To summarize, our perspective proposes that generalization is an emergent phenomenon that involves recurrent similarity computation mediated by the dynamic interaction of multiple stored conjunctive experiences. We also offer a simple mechanism to account for the evolution of a capacity for inference, based on the strengthening of individual premise pair memories through training. We next consider a prominent empirical finding in the literature that would appear to pose a challenge to our “weight strengthening” account of inference, specifically that patients with amnesia show relatively intact performance in classification of novel exemplars of a category, a form of generalization, even though they show markedly impaired recognition performance (Knowlton & Squire, 1993).

Empirical Results: Knowlton and Squire (1993)

Twelve control subjects and 10 amnesic subjects (five with medial temporal lobe damage, five with diencephalic damage) performed a categorization task and a recognition memory task (Knowlton & Squire, 1993). During the study phase of the categorization task, subjects viewed 40 dot patterns that were high distortions of a prototype (Posner & Keele, 1968). During the test phase, they viewed four repetitions of the prototype dot pattern, 20 new low distortions, and 40 random patterns and were asked to judge whether each test pattern belonged to the same category as the patterns viewed during the study phase. In the recognition experiment, subjects viewed five random dot patterns, presented eight times each. At test, they were shown the five previously studied patterns, as well as five new random patterns, and were asked to judge whether each test pattern was old or new.

The results from the empirical study are illustrated in Figure 12 (upper panels) Two features are apparent: First, both control and amnesic subjects showed the prototypicality effect during categorization, endorsing the prototype pattern, low/high distortions, and random patterns with decreasing frequency (see Figure 12, upper right panel). Second, patients with amnesia were markedly impaired at the recognition memory test but did not perform significantly differently from control subjects at categorization (see Figure 12, upper left panel), with a significant interaction between subject group and task reported.

Simulation

We adopt a procedure that shares principles with the application of the GCM model to this data set (see Nosofsky & Zaki, 1998, for details). We did not use dot pattern stimuli but chose feature patterns that approximated the patterns of rated similarities among dot patterns of different types (Nosofsky & Zaki, 1998, Table 2, p. 251). For the categorization task, we used four studied items, each one-bit distortions (i.e., 211111, 121111, 112111, 111211) of a six element prototype (i.e., 111111). Test patterns consisted of the prototype, a new one-bit distortion (i.e., 111121), and a “random pattern” (i.e., 111122) that differed from each of the studied items by three bits. We used a single stored pattern for the recognition memory task (i.e., 1111111). Two test patterns were used in the recognition simulation: a target (i.e., 111111) and a foil (i.e., 111222), differing from the studied pattern by three bits.

Of note, our use of only one level of distortion in the categorization task during testing followed Nosofsky and Zaki's (1998) inter-item similarity assessments mentioned above. This showed that participants afforded equal ratings when asked to judge the similarity of both low and high distortion test items to previously seen high distortion items presented during the study phase. In contrast to Nosofsky and Zaki's implementation, however, we did not directly use these similarities in our analysis—and the self-similarity of items to themselves was not considered as a free parameter in the model as it was in this earlier work. In our simulation, when a test item was identical to a study item, the two items simply shared all the same features—more generally, we used the set of patterns described above to capture other levels of similarity between study and test items and to produce the approximate equivalent of the rated similarity values reported by Nosofsky and Zaki (1998, Table 2, p. 251).

Model Specifics

The general architecture of the model consisted of recurrence between six featural layers and one conjunctive layer. In the case of the categorization task, the conjunctive layer consisted of four units corresponding to each of the training exemplars. For the recognition task, the conjunctive layer consisted of a single unit. As in previous applications, the network was considered to have stored studied examples prior to testing. Testing was implemented in the model by presenting external input to the relevant units in the feature layer. A hedged softmax activation function, including the parameter C, was used in the conjunctive layer, and a standard softmax function was used over featural layers in the network. Network temperature was set to 1 throughout this simulation, and the regulatory C parameter was set at a level that produced unbiased responding (i.e., equal proportion of hits and correct rejections). As in Nosofsky and Zaki (1998), this resulted in C parameter settings that differed across the two subject groups and tasks. The probability of endorsing a test item as a category member, or of judging an item to be old in the recognition task, was determined directly from summed overall activity in the conjunctive layer. The difference between the performance of amnesics and control subjects was simulated by a change in the magnitude of connection weights in the network.

Simulation Results

As illustrated in Figure 12, we show that REMERGE is able to reproduce the empirical findings of Knowlton and Squire (1993), based on a difference in weight strength between amnesic and control subjects (i.e., 0.83 vs. 1.50, respectively; see Figure 12 for details of other parameters). As such, the network exhibits relatively preserved categorization performance, but impaired recognition performance, at the lower weight strength (i.e., 0.83), thereby simulating the pattern of findings observed in amnesia.

Figure 13 illustrates how the network's generalization and recognition performance vary as a function of the connection weight strength. It can be seen that generalization rises gradually with weight strength, but recognition performance rises more steeply, thereby explaining how amnesics may perform relatively similarly to control subjects on the categorization task but show a more marked impairment at recognition. It is apparent, therefore, that our model is unable to account for the ability of patient E.P. to perform normally at the categorization task but show chance levels of recognition memory performance (Squire & Knowlton, 1995). While the explanation for this interesting finding remains subject to debate, it has been suggested that successful performance on the categorization, but not the recognition, task may be mediated solely by successful retention in working memory of immediately preceding test items (Palmeri & Flanery, 1999).

Discussion of Findings

We have shown, therefore, that REMERGE is able to reproduce the empirical finding observed by Knowlton and Squire (1993) that patients with amnesia show an impairment in recognition memory despite relatively preserved categorization performance. Interestingly, however, in other settings (e.g., the transitive inference paradigm) we have emphasized that a reduction in the strength of premise pair memories in the network has a relatively greater impact on generalization performance—a profile of performance that contrasts with the greater decrement in recognition memory performance (cf. categorization) observed here.

These simulations, therefore, highlight the different types of generalization involved in tasks such as the transitive inference paradigm (and other inferential tasks) and categorization tasks—with only the former requiring the process of recurrent similarity computation, a process that itself depends on the faithful recall of robustly encoded studied (i.e., premise) experiences. Consistent with this notion, recurrence is not critical to simulating the categorization data obtained in Knowlton and Squire's (1993) experiment—either in GCM, which lacks such a mechanism (Nosofsky & Zaki, 1998), or in an implementation of REMERGE where this function was disabled (data not shown). Instead, successful generalization in this setting may be mediated by the initial weak activation of multiple conjunctive units as a function of their direct, input-based similarity to the test pattern being presented, in a fashion analogous to that performed by classical exemplar models (e.g., GCM).

Taken with the available empirical data (see for reviews: Eichenbaum, 2004; Zeithamova et al., 2012), therefore, we suggest that transitivity, related inferential capacities, and recurrent similarity computation are reliant on the operation of an intact hippocampal system. In contrast, we leave open the much debated question as to whether intact categorization performance—and the underlying process of direct, input-based similarity computation—relies critically upon the hippocampus (e.g., see Squire, Stark, & Clark, 2004; Zaki, 2004), given the lack of decisive empirical data in this regard.

Generalized Replay Activity in the Hippocampus Through Recurrency

In the next section, we consider whether REMERGE may also provide a framework for capturing generalization-related phenomena that have been observed in recent sleep studies (Ellenbogen et al., 2007; Gupta et al., 2010). We previously illustrated how generalization at a behavioral level was driven by graded activation of related elements and conjunctive units in the network. We also introduced the suggestion that graded patterns of activity in the network that support generalization can also be related to neural signals measured by fMRI. Here, we suggest that a recurrent view of this circuit also makes novel predictions concerning the nature, and function, of activity patterns that one would expect to be spontaneously generated during offline periods, such as rest and sleep (e.g., slow-wave sleep [SWS]; Buzsáki, 1989; Diekelmann & Born, 2010; Ego-Stengel & Wilson, 2010; Foster & Wilson, 2006; Girardeau et al., 2009; Gupta et al., 2010; Lee & Wilson, 2002; McClelland et al., 1995; Wilson & McNaughton, 1994).

Recent work has devoted much attention to understanding how patterns of activity in the hippocampus differ between waking and resting periods such as SWS, made possible through the availability of multielectrode recording techniques in rats (reviewed in O'Neill, Pleydell-Bouverie, Dupret, & Csicsvari, 2010). It is known that local field potentials are quite different between waking exploration and resting states, the latter characterized by so-called sharp wave ripple (SWR) complexes, comprised of negative sharp waves and ripple complexes in the CA1 region. Hippocampal ensemble activity during SWS is thought to be shaped by the patterns of synaptic weights formed by experience within the CA3 region and between the CA3 and CA1 regions (O'Neill et al., 2010). In this way, synchronous discharges initiated in CA3, perhaps triggered by neocortical oscillations (“up-states”) are viewed to trigger ripples in CA1, whereby waking experience is replayed in time compressed fashion (over 100 ms). Several studies have demonstrated that ripple activity in CA1 reflects the reactivation of recent experiences in linear track environments, expressed as the time compressed sequential firing of location-specific place cells (Foster & Wilson, 2006; Gupta et al., 2010; Lee & Wilson, 2002; Skaggs & McNaughton, 1996). Replay has been shown to occur both in forward (e.g., A–F) and backward (F–A) sequences under such conditions. Ripple activity of this nature is thought to be important for the strengthening of existing memory representations within the hippocampal system itself (Buzsáki, 1989, 1996) and the consolidation of recent information from the hippocampal system to neocortical systems for long term storage (Buzsáki, 1989; Káli & Dayan, 2004; McClelland et al., 1995; O'Neill et al., 2010). In line with the latter notion, ripple activity in CA1 is correlated with activity in neocortical target regions (Ji & Wilson, 2007), and the specific disruption of ripple activity in the CA1 region has been shown to produce an impairment in the consolidation of spatial memories in rats (Ego-Stengel & Wilson, 2010; Girardeau et al., 2009).

A conventional unidirectional view of the hippocampal system suggests that replay activity in CA1 arises due to the reactivation of a CA3 ensemble coding for a single conjunctive experience (e.g., sequence of places visited) and is transmitted via the ERC to neocortical regions as part of the process of consolidation (McClelland et al., 1995; O'Neill et al., 2010; Squire, Cohen, & Nadel, 1984). According to this account, ripple activity in CA1 should reflect the replay of individual episodes, as indeed has often been observed (O'Neill et al., 2010).

Our model, however, which places recurrency as central to an understanding of the hippocampal system, makes very different predictions about the nature and function of replay activity. Specifically, the model provides for the possibility that the hippocampal system may give rise to replay activity through a recurrent activation process involving the interaction of multiple related conjunctive units, rather than simply reflecting the output of single episodes. As such, our model predicts that at least under certain conditions (e.g., low inhibition [i.e., high temperature] states during sleep; cf. Buzsáki, 1989), hippocampal replay should be “generalized,” which we believe has important consequences for the putative functions of replay in aiding neocortical learning (see later).

Empirical Findings: Gupta et al. (2010)

A recent experiment by Gupta et al. (2010) provides some support for the hypothesis that replay in CA1 can result from the combination of related episodes, rather than a single episode alone. Gupta et al. used a richer maze environment, rather than a standard linear track environment that may provide little opportunity for useful generalization across related episodes. A two-choice T maze was employed (see Figure 14, upper panel), where rats were trained to run in one direction around the maze. Rewards were provided in the return arm of the maze, for L, R, or alternating turns at the final choice point, depending on the experimental session. Replay activity was observed in CA1, using multielectrode recording techniques during periods when rats rested and consumed rewards. As in previous studies, CA1 ripples were observed, reflecting the time compressed replay of sequential place cell activity coding actual paths traversed by the rat during recent and remote waking experience. Critically, however, Gupta et al. observed that never experienced shortcut journeys, for instance a path from top left of the maze to top right, were also replayed within a proportion of SWR complexes (see Figure 14, lower panel). These findings, in conjunction with control analyses demonstrating that the proportion of novel shortcut sequences was significantly greater than would be expected by the chance recombination of individual forward and backward replays, suggest that replay activity in CA1 may reflect the combination of multiple episodes, rather than exclusively individual journeys.

The goal of our simulation was to provide support for the notion that spontaneous activity within a recurrent hippocampal system naturally produces shortcut sequences of the sort observed by Gupta et al. (2010), mediated by the interaction of multiple related conjunctive units.

Details of Model Architecture and Replay Simulation

The generic bidirectionally connected two-layer (feature, conjunctive) model formed the basis for the replay simulations. Units in the feature layer corresponded to significant locations in the maze (i.e., Locations 1–12; see Figure 14), with conjunctive units corresponding to pairwise journeys (e.g., 1–2). As before, weights were fixed, and the network was considered to have stored paths corresponding to paths taken during waking experience. These were coded as pairwise journeys, as positive bidirectional connections between feature layer units (e.g., 1, 2) and a conjunctive unit (1–2). Weights leading from a feature unit corresponding to the start of a journey (e.g., 1) to the relevant conjunctive unit (1–2) were greater (i.e., weight strength = 1) than those leading from the feature unit corresponding to the end (e.g., 2; i.e., weight strength = 0.9). This weight asymmetry resulted in a tendency for the network to replay paths in a forward, rather than reverse direction, consistent with prior empirical data (O'Neill et al., 2010). A logistic sigmoid activation function was used in the feature layer, and a hedged softmax activation function was used over the conjunctive layer. The two free parameters in the network (τ, C) were set at 0.2 and 5, respectively. In order to produce variability in path generation, a small amount of random noise (M = 0, SD = 0.005) was added to the net input to units.

Replay in our model occurred as follows: (1) The spontaneous activation of pyramidal cells in the hippocampus during SWS states was simulated, as in (Johnson & Redish, 2005), by randomly selecting one unit in the feature layer and providing it with an external input of 1 (e.g., unit 4); (2) the network was allowed to cycle until another unit (except the one receiving external input) crossed a preset threshold (i.e., activation value of 0.6). If more than one unit did so simultaneously, one unit was chosen at random. The winning unit (e.g., 5) was given an external input of 1, and the network allowed to cycle once again. Importantly, the previously active unit (i.e., 4) was prevented from becoming active, in effect being inhibited from participating in the rest of the replay sequence, analogous to an inhibition of return mechanism. This procedure was iterated until a replay sequence of up to three nodes was generated (e.g., 4–5–9). Path lengths of 3 were chosen to mirror the approximate lengths of paths reported by Gupta et al. (2010).

It is important to acknowledge that our model (and previous work; e.g., Johnson & Redish, 2005; also see Molter, Sato, & Yamaguchi, 2007) followed a simplified scheme, designed to mirror the broad characteristics of replay rather than obey specific neurophysiological properties of the hippocampus. As such, we “hard-coded” paths in the model, rather than simulating the actual encoding of paths through the environment, often implemented under theta state conditions facilitating sequence coding in the CA3 region. Further, paths were coded in pairwise, rather than sequential, fashion, both for simplicity and to mirror the coding of premise pairs in previous simulations. Given the inherent limitations of our model, therefore, we aimed to simulate the overall pattern of findings observed by Gupta et al. (2010) rather than fit the detailed quantitative pattern of the data.

Replay sequences were generated using this operating procedure, for a total of 50 instances from each possible starting position in the maze (i.e., unit in the feature layer). As in Gupta et al. (2010), replays were categorized according to whether they reflected individual experiences (forward [e.g., 4–5–6], backward [4–3–2]), shortcut sequences (e.g., 8–4–5), or disjoint sequences (e.g., 4–5–9). The model generated 60% forward replay sequences, 26% backward sequences, 14% shortcut sequences, and <1% disjoint sequences. We were therefore able to replicate the basic finding of shortcut sequences observed by Gupta et al. (2010), though in a greater proportion than observed empirically. This difference may have arisen for a variety of different reasons (e.g., strength of weights in the network): As noted above, our aim was to confirm the basic capacity of the network to generate novel shortcut sequences during replay, rather than achieve a close fit to the empirical data through optimizing the parameters of the model.

Summary of Model Simulations and Conclusions

1. Spontaneous activity in the hippocampus during offline periods is generally thought to reflect the reactivation of a single CA3 ensemble coding for an individual episode. Here, we verify the basic capacity of a recurrent network to produce what we call “generalized” replay.

2. We describe a simulation of the model that produces patterns of replay activity reflecting never-experienced shortcut paths, replicating the basic finding observed in a recent empirical study by Gupta et al. (2010).

3. In previous simulations, we illustrated how recurrency, through the co-activation of multiple conjunctive units for related episodes, could support generalization at a behavioral level (e.g., in the transitive inference task). It is worth noting that while the set of network parameters (e.g., temperature) employed in the current replay simulations resulted in the co-activation of multiple conjunctive units, generalized replay could in principle be mediated by a “chaining” process involving the sequential activation of individual conjunctive units.

In addition to simplifications alluded to above (e.g., coding of sequential journeys in pairwise fashion in the model), we also wish to point out that our model was only designed to simulate shortcut paths that reflect combinations of previously taken journeys—as observed by Gupta et al. (2010). In contrast, the ability of rats to navigate to a particular location (e.g., hidden platform in a water maze; e.g., Eichenbaum, Stewart, & Morris, 1990) from a range of novel starting points falls outside the scope of our simulations, perhaps reflecting the use of an allocentric map of space based on specifically geometric computations (O'Keefe & Nadel, 1978). In a later section (see General Discussion), we outline empirical predictions that derive from our notion of generalized replay activity in the hippocampal system—and we discuss how this may have broad implications for the function of the hippocampal–neocortical dialogue.

The Emergence of a Capacity for Transitive Inference Over an Offline Delay: Ellenbogen et al. (2007)

We next examined the intriguing finding that transitivity emerged over an offline delay during which subjects rested or slept, without receiving further additional training (Ellenbogen et al., 2007). We first consider whether this striking enhancement of inferential performance over an offline delay might also be accounted for by the strengthening of premise pair memories—as we previously suggested could mediate the effect of continued training in a previously outlined simulation of the transitive inference task (i.e., in Ryan et al., 2009; see Figure 5).

Fifty-six healthy volunteers participated in the experiment by Ellenbogen et al. (2007). There were two principal subject groups of interest, defined by the offline period (20 min, 12 hr) that intervened between premise pair training (e.g., A–B) and testing on inference pairs (e.g., B–D, B–E, etc.). The stimuli and design of the study were similar to Ryan et al. (2009), though a different training regime was used. During training, premise pairs were presented in pseudorandom order (e.g., B–C could not follow A–B), a manipulation designed to avoid revealing the structure of the hierarchy to subjects. Premise pair training continued until subjects reached a predefined criterion, which was set at greater than 75% on the middle premise pairs (i.e., B–C, etc.). Subjects then completed an immediate test on the premise pairs, without feedback, followed by a variable offline delay (e.g., 20 min or 12 hr). Following this delay, subjects were tested with inference pairs and premise pairs, once again without feedback. Subjects were asked to report their confidence in their choices during inference trials, though explicit awareness of the hierarchy was not assessed in this study.

As shown in Figure 15 (upper panel), despite successful performance on premise pairs (e.g., B–C) after the initial training session (c90%), inferential performance was not significantly different from chance levels, shortly after the completion of training (i.e., following a 20-min offline delay). Strikingly, a capacity for transitivity emerged after an offline delay of 12 hr (or more): Inferential performance, averaged across both “close” (i.e., B–D, C–E) and “distant” (i.e., B–E) inference pairs, was 75%.

The results of our simulation of this experiment are shown in Figure 15 (lower panel). As such, it can be seen that the network is able to simulate the empirical finding that a capacity for inference appears only after an offline delay with little change in premise performance. Notably, the network is able to reproduce this phenomenon by appealing to a simple mechanism by which the strength of premise pair memories increases over the offline delay—analogous to the mechanism previously proposed to underlie the effects of continued training. As discussed previously, the large lag generalization effect observed in this simulation arises as a consequence of the relatively low network temperature (i.e., 0.2; see Figure 4, lower panel).

While the network is able to simulate the sudden emergence of inferential capacity over an offline delay, two points suggest that such a weight strengthening account falls short of providing a complete account of such generalization-related sleep phenomena. First, the network fails to reproduce the more specific benefit of sleep on inferential capacity reported by Ellenbogen et al. (2007), whereby performance on distant pairs (i.e., B–E) was seen to improve to a greater extent compared to performance on close pairs (i.e., B–D, C–E; empirical data not shown). Indeed our model, implementing recurrent similarity based generalization, is unable to provide a robust account for this phenomenon (i.e., B–E > B–D). Specifically, performance on close inference pairs in our model tends to emerge at lower, or similar, weight strengths, compared to that of distant inference pairs, depending on the exact parameter settings (i.e., temperature, beta: results not shown).

Second, it has often been argued that offline periods result in mnemonic benefits (e.g., in paired associate paradigms) through a reduction in forgetting (Diekelmann & Born, 2010; Wixted, 2004). Indeed, it is typically the case that memory performance, for instance on a paired associate recall task, is actually worse after a period of sleep compared to immediately after training, though significantly superior to a comparable period of waking activity (though see Tucker & Fishbein, 2008, for an example of better memory after sleep compared to immediately post-training). Notably, the simulation above relies on the idea that sleep leads to the strengthening of recent memories. While recent work suggests that performance on instrumental tasks involving rewards may actually be enhanced as a result of sleep—a situation that has some similarities to feedback-based learning tasks such as the transitive inference paradigm—it remains uncertain whether such an effect would generalize to the kind of experimental scenario considered here (Brawn, Nusbaum, & Margoliash, 2010).

How then might the beneficial effect of an offline delay on the emergence of a capacity for inference be accounted for? One hypothesis, favored by Ellenbogen et al. (2007), is that offline delays may facilitate the creation of stored generalizations (e.g., coding the linear structure of the transitivity task) through the recombination of multiple related episodes, which can then be used as the basis of performance on inference trials. In the previous section, we presented empirical evidence suggesting that replay activity during offline periods may indeed reflect the recombination of related episodes and therefore be generalized in nature, and we demonstrated that REMERGE is able to reproduce this basic phenomenon. While we do not formally explore the putative creation of stored generalizations in the current article, given our use of an idealized learning scheme and the potential pitfalls of allowing new learning to coexist with memory retrieval (e.g., Hasselmo, 1993), we believe the recurrent architecture we advocate constitutes a natural framework in which to consider these ideas (see General Discussion).

Overall Summary

Empirical evidence supports an important function for the hippocampus in generalization, in addition to its well-established role in episodic memory (Eichenbaum, 2004; Zeithamova et al., 2012). Here, we ask how the hippocampus mediates generalization, given that this brain structure, and in particular the DG and CA3 subregions, is typically viewed to perform pattern separation (i.e., orthogonalization) of incoming inputs—a process that emphasizes the differences between related experiences at the cost of disregarding their shared features. Our approach to this issue involves “big-loop” recurrency within the hippocampal system, or more widely between the hippocampal system and the neocortex, which mediates the interaction of pattern separated conjunctive codes (broadly ascribed to the CA3 region) with componential representations viewed to be instantiated in regions of the neocortex. As such, generalization in the REMERGE model occurs at the stage of memory retrieval and results from recurrent similarity computation, with the activity of conjunctive units reflecting their similarity to both externally presented sensory inputs, and inputs reconstructed by the network. Recurrence, therefore, expands the scope of exemplar and similarity based models of generalization to include inferential paradigms such as the transitivity task, while preserving the abilities of these models to perform categorization and recognition memory.

We also offer a mechanistic account of how a capacity for generalization may emerge through training (e.g., in Ryan et al., 2009)—or differ between subjects (e.g., Shohamy & Wagner, 2008)—simply through the strengthening of existing memories for premise items, rather than necessitating the creation of explicit representations of the task structure. Our theory, therefore, offers a formal model of how hippocampal generalization emerges, as if within a virtual memory space as previously hypothesized (Eichenbaum et al., 1999), but where the linkage of related experiences occurs on the fly through recurrent interactions, rather than being statically engrained in the actual overlap of neuronal codes at the representational level. As such, the current perspective may help reconcile fundamental differences between two highly influential views of memory: the CLS (McClelland et al., 1995) and relational theory (Cohen & Eichenbaum, 1993; Eichenbaum, 2004).

In this section, we discuss the relationship between our retrieval-based theory of generalization and encoding-based overlap accounts that emphasize the importance of creating overlapping representations for related experiences during training (i.e., encoding). We first consider the relationship between our account and a formal encoding-based overlap theory: the temporal context model (TCM; Howard et al., 2005). We then turn to points of contact between our account and the relational theory of hippocampal function (Cohen & Eichenbaum, 1993; Eichenbaum, 1999), before examining how our theory fits with the empirical observation that hippocampal neurons have been observed to respond to common features across multiple experiences (i.e., “nodal codings”). Next, we outline predictions that arise from our notion of generalized replay in the hippocampal system. Finally, we discuss implications for the individual contributions of the hippocampus and neocortex to semantic learning, construed as learning the general structure of the environment.

REMERGE and TCM: Key Differences and Divergent Predictions

We first examine the relationship between REMERGE and the TCM, a formal mechanistic account that proposes that it is the overlap between hippocampal representations for related stimuli (or experiences) that is critical to generalization (Howard et al., 2005). TCM was originally developed to account for essential properties of behavioral data on tasks involving free recall (Kahana, 1996; Polyn & Kahana, 2008; Polyn, Norman, & Kahana, 2009) but has more recently been proposed to account for inferential behavior in the transitive inference task (Howard et al., 2005). Briefly, TCM argues that contextual states, rather than item–item associations, act as the primary cues for recall of items. As such, items are retrieved as a function of their similarity to the current state of context, which in turn is influenced by both the items themselves and a general tendency to drift over time.

To summarize the essential principle of the mechanism of generalization by TCM and to illustrate how this differs from that of REMERGE, we use the transitive inference paradigm as an illustrative scenario. During training, the contextual representation of each item in a premise pair (e.g., Items A and B) shares common features due to their temporal co-occurrence. Critically, the item-contextual codes of non-adjacent items—for example, A and C—also overlap: The reason for this is that during BC training, the C item is bound to the current state of context, which includes the retrieval of the context in which B was encoded (i.e., co-occurrence with A). Over training, therefore, the model develops a similarity structure that comes to reflect the distance of items within the linear hierarchy of the transitive inference task. While to our knowledge, simulations of the model's behavioral performance on the transitive inference paradigm have not been presented, it is argued that successful inference could be mediated by a value gradient (i.e., B > E) constructed from TCM's linear representation of items. Further, it is proposed that the hippocampus is critical to transitivity judgments through its capacity to support new item-contextual learning.

TCM, therefore, exhibits the phenomenon of transitivity in a qualitatively different way from REMERGE. While inference in REMERGE can be considered an emergent phenomenon at retrieval mediated by recurrency operating over pattern separated conjunctive codes, the same inference in TCM is dependent on the creation at encoding of overlapping item-contextual codes whose similarity to one another reflects the relationship of items in a set of related experiences. Here, we focus on this key difference between REMERGE and TCM that parallels a distinction drawn between model-based and model-free controllers influencing behavioral choice in situations involving reward learning (e.g., Daw, Niv, & Dayan, 2005). In a later section, we consider possible differences between REMERGE and TCM in terms of predicted content of hippocampal replay activity during offline periods.

Model-free systems typically store cached values acquired through learning, for example, concerning the value of stimuli or actions divorced from their respective outcomes (Daw et al., 2005). In contrast, model-based systems construct predictions on the fly, effectively at the point of choice, through a retrieval-based process of search involving the chaining together of short-term predictions concerning the outcome of individual actions in a sequence. While model-free systems are computationally efficient, they are relatively inflexible and tend to resist the updating of their representations in the face of important changes in the structure of the environment (cf. model-based systems). The REMERGE network bears similarities to a model-based system, given that generalization emerges through a retrieval-based search-like process over the space of conjunctive experiences mediated by recurrency. TCM, in contrast, can be construed in broad terms as a model-free system, though it should be noted that TCM learns item representations that are useful for generalization, rather than cached values. Expressing the difference between REMERGE and TCM in these terms may be useful in suggesting possible empirical avenues for distinguishing between them, a possibility we consider next.

REMERGE and TCM make different predictions concerning the effects of changing the structure of the environment on subjects' behavior in the transitive inference and other related tasks. While it may be difficult to definitely distinguish between these competing models of generalization on the basis of any single experiment, it is potentially informative to consider how subjects would respond in the following scenario: The initial phases of the experiment would be identical to a typical five-pair transitive inference task. Next, subjects would receive training on a new premise pair, where F should be chosen over A (i.e., effectively changing the structure of the set of premise pairs from a linear hierarchy to a circle). Under these conditions, REMERGE makes the prediction, which we verify in an additional simulation (see Figure 16), that subjects would be equally likely to choose B over E in a subsequent test trial, given its retrieval-based similarity mechanism of generalization. In contrast, TCM, along with value transfer mechanisms (Frank et al., 2003; von Fersen et al., 1991), would suggest that subjects should continue to favor item B. In TCM, this would be the case because learning of a novel premise pair (i.e., F+ A−) would not generally be expected to change previously encoded item-contextual representations associated with other items (e.g., of B). Of note, both REMERGE and TCM would predict that subjects would continue to favor B over D in a B–D trial, albeit through different mechanisms. While preliminary investigations in chimpanzees (Gillan, 1981) and rats (Davis, 1992) involving small numbers of experimental subjects suggest that transitive responding may indeed be disrupted by the addition of a new inconsistent premise pair (e.g., F+ A−), more extensive testing would be potentially illuminating in distinguishing between these two models of transitivity behavior.

REMERGE and the Relational Theory

Our perspective helps reconcile key differences between two highly influential views of memory—the CLS (McClelland et al., 1995) and relational theories (Cohen & Eichenbaum, 1993; Eichenbaum, 2004)—that to our knowledge have rarely been explicitly acknowledged in the literature to date. CLS theory, in its original form, prioritized the role of the hippocampus in episodic memory instantiated in a pattern separated representation scheme, at the expense of generalization capacity. In contrast, the relational theory (Cohen & Eichenbaum, 1993) has long regarded generalization and semantic learning to be a fundamental aspect of hippocampal function, without such a clear focus on the nature of computational constraints that support optimal performance (e.g., capacity) in a neural system widely agreed to support episodic memory. Indeed, recent formulations of the relational theory, drawing on the empirical phenomenon of “nodal codings” (see below; also see Eichenbaum et al., 1999), have argued that the networking of related episodic memories—based on their overlap at the representational level—is central to hippocampal processing and critical for generalization (Eichenbaum et al., 1999).

The present account of the hippocampal system, instantiated in the REMERGE model, can be considered to marry essential insights provided by a relational view of memory (i.e., compositionality of episodic memories; generalization through linking of related episodes within a memory space; Cohen & Eichenbaum, 1993; Eichenbaum et al., 1999; also see Buzsáki, 2005) with the computational constraints of an episodic memory system imposed by CLS theory and allied viewpoints (i.e., rapid learning, pattern separated representations; Burgess, 2006; Marr, 1971; McClelland et al., 1995; McNaughton & Morris, 1987; O'Reilly & Rudy, 2001; Rolls & Kesner, 2006). This synthesis arises through exploiting the principle of recurrency, which affords the network a capacity to generalize efficiently but critically still preserves the essential characteristics of pattern separated codes in the hippocampus. The account we offer, therefore, seeks to preserve the foundational division between representational schemes employed by the hippocampus (pattern separated) and neocortex (overlapping codes for related experiences; Marr, 1971; McClelland et al., 1995) but considerably expands the potential contribution of the former to generalization, by incorporating recurrent flow within the circuit. In this way, we have been able to capture empirical data implicating the hippocampus in tasks involving generalization (Eichenbaum, 2004; Zeithamova et al., 2012), which until now have tended to speak exclusively to the relational theory, residing as they have done largely outside the explanatory framework of CLS theory.

Pattern Separation Versus Nodal Codings?

Our approach emphasizes the importance of considering how the hippocampus might contribute to generalization based on pattern separated neural codes for conjunctive experiences. This contrasts with encoding-based overlap accounts, including the TCM and relational theory (Eichenbaum, 1999; Gluck & Myers, 1993; Howard et al., 2005; also see O'Reilly & Rudy, 2001). These accounts embrace the notion that it is the overlap between representations of similar episodes (or stimuli) that is critical to inference (e.g., in the transitivity task). We next consider empirical evidence that has been interpreted as supporting this argument, and we ask whether such observations are consistent with our perspective.

In one important study (Wood, Dudchenko, & Eichenbaum, 1999; also see Eichenbaum, 1999; Ferbinteanu & Shapiro, 2003; Moita, Rosis, Zhou, LeDoux, & Blair, 2003), activity of neurons in the hippocampus was recorded while rats performed a continuous non-match-to-sample task. They observed hippocampal neurons with a range of specificities, varying from some, termed nodal codings that responded to a common feature across many events (e.g., a particular odor), to others that showed more selective conjunctive responses (e.g., to a non-match decision performed in relation to a particular odor at a particular place). How then can the presence of hippocampal nodal cells encoding common features be accommodated by our perspective and related perspectives (e.g., Marr, 1971; McNaughton & Morris, 1987; Treves & Rolls, 1992), espousing an important role for the hippocampus in pattern separation?

Two points are worth highlighting in the current context: First, in the current perspective, we adopt a theoretical ideal of pattern separation (i.e., zero overlap of conjunctive codes) to illustrate how generalization can still be achieved through recurrency under these circumstances. We naturally accept, however, that neural representations of similar episodes (or environments) are likely to overlap since of course in reality the process of pattern separation is far from perfect. Such “residual” overlap of neural codes within the CA3 region might be viewed to arise from a failure of optimal pattern separation of incoming inputs that already overlap at the level of upstream regions (i.e., ERC), rather than a specific hippocampal mechanism designed to support generalization per se. Nevertheless, it is worth bearing in mind that residual overlap of this nature may support a degree of “short-range” generalization, even in the absence of recurrency. For example, it has been shown that successful B–D, but not B–E, test trial performance in the transitivity task may be mediated by such a mechanism (O'Reilly and Rudy, 2001).

Second, it should be emphasized that pattern separation is most appropriately thought of as a function of part of the hippocampal circuit, rather than the system as a whole (McNaughton & Morris, 1987; O'Reilly & McClelland, 1994). As discussed previously, it is generally agreed that information processing within the DG and CA3 regions is particularly important for creating and sustaining orthogonalized codes for similar input patterns. This view is supported by empirical data suggesting that neural codes for similar environments in DG and CA3 overlap less than those in CA1. In a recent experiment (S. Leutgeb et al., 2004), neural activity was recorded in both CA3 and CA1 while rats foraged in different enclosures (e.g., square, circle) in one of three different rooms (A, B, C). Importantly, they noted a striking difference between CA3 and CA1 in terms of the neural representations observed in each of the different room configurations. While the pattern of neural activity in CA3 was essentially orthogonal in each environmental setting, CA1 responded to common features, analogous to the nodal codings reported by Wood et al. (1999): For example, the pattern of CA1 activity in the square enclosure was highly correlated (i.e., similar) regardless of the room in which it was situated.

This evidence suggests that neural codes in DG-CA3 may be viewed as more conjunctive and pattern separated, compared to those in CA1 that are correspondingly more overlapping and componential. Related evidence using similar experimental settings is also consistent with this conclusion (J. K. Leutgeb et al., 2007; Skaggs & McNaughton, 1998; Tanila, 1999). While it is not clear if the nodal codings (e.g., odor cells) observed by Wood et al. (1999) tended to be situated in the CA1 region, with more specific codings in CA3 (e.g., decision-odor-place codings), such a pattern of findings would be consistent with this overall scheme.

It is also worth asking how hippocampal neurons (e.g., in CA1) that code for common features of an environment might arise, and what their functional significance might be. One hypothesis is that nodal codings are driven by feedforward connections from the ERC to CA1, as proposed by McClelland and Goddard (1996). While further empirical work is necessary, such a mechanism has been suggested to be consistent with the rapid timecourse (cf. CA3) of the development of CA1 representations in S. Leutgeb et al.'s (2004) study. If this were the case, this might suggest that nodal codings in the hippocampus do not necessarily have any special functional role in generalization (i.e., as suggested by the relational theory). Instead, they might reflect the use of a coding scheme in CA1 that is relatively more componential than that sustained by CA3 to allow information arriving from CA3 (e.g., as hippocampal output during waking, or during replay states) to be transformed into a format that can be interpreted by other brain regions (McClelland & Goddard, 1996).

Our suggestion, therefore, is that nodal codings in the rodent hippocampus may reflect the componential nature of inputs (and outputs) of the DG–CA3 regions, rather than being indicative of a specific hippocampal mechanism for generalization per se. As noted previously, we believe that structured inputs/outputs of this nature are critical to the rapid creation of episodic representations comprised of arbitrary conjunctions of multiple components (e.g., see McClelland & Goddard, 1996)—with generalization then relying on a recurrent mechanism that operates over the set of conjunctive codes. In accordance with this general position, we broadly identify the conjunctive layer of our model to the type of codes sustained by the DG–CA3 regions, and the feature layer to the CA1 and upstream neocortical regions including the ERC. We note, however, that it is possible that neurons sustaining conjunctive codes may interdigitate with those supporting relatively more componential codes, rather than being confined to anatomically distinct subregions.

At this stage, it is also worth drawing attention to a type of hippocampal neuronal response profile that bears some similarities to the nodal codings described above. Neurons in the hippocampi of epilepsy patients have been found that appear to respond to stimuli that denote a particular individual (e.g., the written name or a picture relating to Jennifer Aniston) and associated entities (e.g., Brad Pitt), rather than being specific for any unique episodic experience (Quiroga et al., 2008; Quiroga, Reddy, Kreiman, Koch, & Fried, 2005). Interestingly, however, neuronal response selectivities of this sort have typically been reported to occur throughout the medial temporal lobe (including the amygdala) and at very long latencies (e.g., 300–600 ms; Quiroga et al., 2008, 2005). While these observations may also be accounted for along the lines discussed previously (i.e., as part of a relatively componential code sustained by regions upstream of CA3, or even interdigitated with more conjunctive neurons in CA3), one additional possibility suggested by their surprisingly long response latencies is that such neurons are not primarily driven by a feedforward mechanism but reflect the operation of a recurrent mechanism involving the co-activation of multiple related conjunctive ensembles. While we note, therefore, that the set of observed findings tends to be consistent with the view that neurons throughout the MTL may respond to abstract entities such as individuals or objects that are present in many different episodes (Quiroga et al., 2008), the presence of such neurons may not be inconsistent with the possibility that generalizations across experiences are formed by recurrent computations involving more conjunctive representations.

Comment on Stored Generalizations

The notion that stored generalizations, often termed integrated representations, are critical to generalization is common to several accounts of inferential behavior in tasks such as the transitive inference, paired associate inference, and acquired equivalence paradigms (Eichenbaum, 2004; Howard et al., 2005; Shohamy & Wagner, 2008; Zeithamova & Preston, 2010). According to the relational theory discussed previously (Cohen & Eichenbaum, 1993; Eichenbaum et al., 1999), it is the overlap in neural representations between related episodes that supports discovery and, presumably, explicit representation of the task structure (e.g., A–B–C…–F: linear hierarchy in the transitive inference task). The integrative encoding hypothesis, in contrast, suggests that it is the pattern completion properties of the hippocampus mediating the retrieval and subsequent encoding of related episodes into a larger representation of the task structure (e.g., linking stimuli F1, F2, S1, S2 in their experiment; e.g., see above), that are critical to inference (Shohamy & Wagner, 2008).

The REMERGE model places particular emphasis on the idea that a capacity for generalization is an emergent property of the recurrent network we describe, effectively acting at the stage of memory retrieval. As such, determining when such stored generalizations, or integrated representations, formed at encoding are needed to account for behavioral performance represents an important challenge for future investigators. That said, we do acknowledge that stored generalizations may well arise during training, testing, and intervening periods including but not limited to sleep, and that they may thereby make a contribution to generalization and inferential behavior. For instance, it seems clear that the capacity to explicitly report details of the task structure (e.g., the linear hierarchy in a transitive inference task) can emerge in human subjects under certain experimental conditions (Moses et al., 2006; C. Smith & Squire, 2005; also see Kumaran et al., 2007, 2009). Such explicit reports and other behavioral indices of generalization could reflect processes that occur as a consequence of recurrent similarity-based generalization as captured in the REMERGE model.

Therefore, we would not wish to exclude the possibility that the hippocampus creates new representations combining elements from distinct experiences that became co-activated through recurrent processing. Indeed, the notion that emergent generalizations once created through a REMERGE-like process would then themselves be stored in memory is very appealing. It will remain for future research to evaluate how easily the storage of such representations could co-exist with the episodic memory function of the hippocampus and related areas. More generally, the potential perils of allowing new learning to be intertwined with network recall should not be underestimated (Hasselmo & Bower, 1992; Hasselmo & Schnell, 1994).

These caveats not withstanding, if weights were allowed to change during network operation during learning or offline periods such as slow-wave sleep (see below), it is reasonable to imagine that, under certain circumstances (i.e., depending on learning algorithm and inhibitory control), a new conjunctive representation capturing the graded patterns of activity present on the feature layer would result in stored generalizations of the sort envisioned by Shohamy and Wagner (2008). Importantly, stored generalizations, created in this manner, would be coded by distinct neural representations that would be simulated using their own localist conjunctive units in the model, separate from those pertaining to the individual premise pairs, thereby limiting the potential detrimental effect on the network's ability to recall specific experiences (i.e., AB).

Generalized Replay in the Hippocampal System: Perspective and Predictions

In line with the ideas considered above, we suggest that generalized replay activity in the hippocampal system may facilitate the creation of new representations (i.e., stored generalizations), formed from the recombination of multiple related episodes—as well as the strengthening of existing hippocampal representations (i.e., intrahippocampal consolidation; Buzsáki, 2005). As such, we draw on previous ideas that replay activity may be an important determinant of plasticity within the hippocampal system itself, by bringing episodes spanning relatively long timescales (e.g., tens of seconds) within a time window appropriate for linking them through associative synaptic modification (e.g., 100 ms; Buzsáki, 1989, 1996, 2005). Interestingly, this general idea has been proposed to underlie the transition from direction-selective, to omnidirectional, place cell responses in the hippocampus (Buzsáki, 2005).

We also suggest the following predictions concerning generalized replay in the hippocampal system:

1. Our model predicts that generalized replay should be observed in a wide range of settings, for example, an offline period following a block of transitive inference task performance. Further, a selective impairment of the potential for recurrency within the hippocampal system would be predicted to abolish generalized replay activity. While existing neuroanatomical evidence is consistent with a potential for recurrency in the hippocampal system (van Strien et al., 2009), further work is required to define more precisely the architecture of the relevant neural circuitry to assess the feasibility of such an experimental manipulation. Of note, one possible explanation for the infrequent detection of generalized replay thus far is the relatively simple structure of rodent tasks in which hippocampal replay activity is typically recorded, which tend to offer little opportunity for recurrent similarity-based generalization given their inherent lack of overlapping experiences in the environment (though see Gupta et al., 2010).

2. In the model, graded patterns of activity in the network emerge with the strengthening of premise pair memories during continued training and underlie both behavioral generalization performance and generalized replay during offline periods. Consequently, generalized replay may serve as a marker of the capacity of the system to support behavioral generalization at a given stage of training. A resulting prediction, therefore, is that replay activity during offline periods should arise in parallel with a capacity for behavioral generalization during waking experience.

3. As noted previously, hippocampal replay activity is thought to play a role in neocortical (McClelland et al., 1995; O'Neill et al., 2010) as well as intrahippocampal learning—both in strengthening existing memory representations for individual episodes (Buzsáki, 1989, 1996) and, we suggest, in facilitating the creation of novel stored generalizations. Our model makes the prediction that generalization performance should be relatively susceptible to the disruption of replay-mediated intrahippocampal learning processes, given the particular sensitivity of this behavioral measure to the strength of premise pair representations in the network as well as the possible influence of stored generalizations on behavioral performance. While recent work has demonstrated the feasibility of detecting ripple activity within the hippocampus and has examined the consequences of its disruption on (spatial) memory (Ego-Stengel & Wilson, 2010; Girardeau et al., 2009), the specific consequences for generalization performance have not yet been explored.

The putative role of generalized replay in facilitating intrahippocampal learning also suggests that subjective reports pertaining to neural activity during sleep may correlate with subsequent generalization performance. While such “dream reports” have been typically linked to REM sleep, they have also been associated with SWS sleep, a phase in which hippocampal replay activity is most prominent (Wamsley, Tucker, Payne, Benavides, & Stickgold, 2010). Our model predicts that the frequency of dream imagery reflecting the recombination of multiple related episodes experienced during waking (e.g., a shortcut through a maze) may correlate with subsequent generalization performance (e.g., shortcuts, transitivity judgments). One recent study reports preliminary evidence consistent with this prediction: The frequency with which subjects reported SWS associated dreaming about a maze like environment in which they had been previously trained showed a significant correlation with subsequent behavioral performance (Wamsley et al., 2010). Interestingly, dream reports in this study tended to be fragmentary, and did not relate to any single journey through the maze. However, they were rather non-specific, precluding any specific conclusions about their relationship to the content of neural replay activity.

Our account links the content of replay activity produced by a recurrent hippocampal system during SWS to a capacity for generalization. Interestingly, however, previous work has suggested that REM sleep, rather than SWS, facilitates generalization (Walker & Stickgold, 2010). Importantly, however, the benefits of REM sleep in terms of generalization have been observed in tasks like the Semantic Priming Test (Stickgold, Scott, Rittenhouse, & Hobson, 1999) and the Remote Association Test (Cai, Mednick, Harrison, Kanady, & Mednick, 2009)—which we consider to be qualitatively different from the transitive inference (and related) tasks and arguably not dependent on the hippocampus (though no data speak to this issue, to our knowledge). In our view, the critical feature of the transitivity task that accounts for its dependence on the hippocampus is the arbitrary nature of the associations learnt (i.e., premise pairs), with inference supported by a capacity to relate multiple premise pairs to one another. This is not the case in tasks such as the remote association test—in the study by Cai et al. (2009), for example, a period of REM sleep was reported to increase the extent to which participants could be primed to identify the word that links a triplet of distantly related associates (e.g., associates: sixteen, heart, candy/linking word = sweet). In this experimental context, REM sleep mediated modifications (e.g., in terms of connectivity) to existing neocortical representations for the familiar stimuli used (e.g., “sixteen” and “sweet”) may be sufficient to mediate performance enhancements (see Cai et al., 2009, for a related argument) without involving the hippocampus.

It is worth noting that representational overlap theories (e.g., TCM) might also make the prediction that replay activity in the hippocampal system during offline periods should be generalized in nature, rather than relating solely to a single experience, and should correlate with behavioral indices of generalization. While to our knowledge this issue has not been considered previously, it is therefore conceivable that overlap-based accounts might make many of the same predictions concerning the function of generalized replay in the hippocampal system. There is a possible divergence of predictions, however: REMERGE predicts that periods of generalized replay may be interspersed between periods of conventional single-episode replay (e.g., AB episode), given that network output is determined by the operating level of inhibition that is likely to vary over the course of an offline delay (cf. Buzsáki, 1989). Overlap-based theories (e.g., TCM) would find it difficult to account for such a pattern of replay activity—this is because the similarity between related episodes (i.e., AB and BC) is viewed to be statically engrained at the representational level, rather than dynamically computed at the point of retrieval as in REMERGE.

Perspective on the Role of the Neocortex in Generalization and Semantic Learning

The neocortex is widely held to be a powerful learner of the general properties of the environment but is thought to have several important limitations (McClelland et al., 1995; Rogers & McClelland, 2004). Learning in neocortical networks, at least for information that is unrelated to or inconsistent with existing knowledge, must necessarily proceed slowly, generally requiring many hundreds or thousands of training trials. This is the case because rapid learning of such information can lead to catastrophic forgetting of previously acquired information (McCloskey & Cohen, 1989). Learning in neocortical networks is also constrained by a requirement for interleaved training, whereby training examples inconsistent with pre-existing structure or encoding a new set of structural relationships must be intermixed with familiar information, to ensure optimal extraction of the structure of the data set and integration with pre-existing knowledge without producing catastrophic interference. Focused learning (repeated presentation of the same new items without interleaving) tends to have the same effect as use of large connection weight changes, also tending to produce catastrophic interference when the new items are inconsistent with what is already known (Grossberg, 1987; McCloskey & Cohen, 1989; see also French, 1999).

The principle of slow learning in the neocortex is particularly problematic given the paucity of learning opportunities in the real world: Consequently, neocortical learning is typically viewed to require help from a complementary learning system, putatively instantiated in the hippocampus, capable of serving as the basis of initial learning and providing additional training trials during offline periods—and additionally acting as a basis for memory for recent episodes (McClelland et al., 1995). Although these provisions certainly help to overcome some of the shortcomings of neocortical learning, several open questions remain, which we suggest might be addressed by recurrency in the hippocampal system.

Proposal of How a Recurrent Hippocampal System May Mitigate Limitations of Neocortical Semantic Learning

1. If neocortical learning is very gradual, how are individuals able to rapidly exploit the relationships between a set of related conjunctive experiences, for example, within a single experimental session? We argue that a recurrent mechanism may underpin the contribution of the hippocampus to generalization in the context of the transitivity and related paradigms. Interestingly, very few studies to date have investigated conceptual learning and generalization in a more naturalistic context (e.g., learning and generalizing about a fictitious set of characters as simulated in McClelland, 1981). We suggest that the hippocampal system may make a particular contribution to generalization in settings where the environment is volatile or experience is limited (i.e., few training exposures), whereas the neocortex may prevail in knowledge representation for information acquired over more extended time periods. The greater the temporal extent, the greater the likely role of neocortical connections, all else being equal.

2. We further suggest that generalized replay from a recurrent hippocampal system may also reduce the computational workload that the neocortex has to perform in order to extract the structure of the environment. Viewed from a geometric perspective, the process of consolidation may be thought of as involving the discovery of lower dimensional components (or manifolds) that underlie a set of related episodic experiences that occupy a high dimensional space. While we emphasize that generalization in REMERGE emerges at retrieval, we tentatively suggest that the operation of recurrency (e.g., during offline periods) may also play a role in deriving more efficient lower dimensional representations of the underlying environmental structure. In the transitive inference task, for example, the goal would be to discover that the items presented during training form a linear structure (e.g., A–B–C–D–E–F). In this respect, it is worth noting similarities between the mechanism of generalization in REMERGE and the workings of the ISOMAP algorithm, a model that has been shown to increase the power of similarity based techniques (e.g., multidimensional scaling) in discovering the low dimensional structure of data sets (Tenenbaum, de Silva, & Langford, 2000). Both models provide a means by which the inherent similarity of two related experiences (e.g., B–C, D–E in the transitive inference task) can be captured, even when pairwise similarities in input-space are uninformative (and in this case equate to zero). REMERGE achieves this through a constraint satisfaction style search of the high dimensional space of episodes performed at retrieval, while the ISOMAP algorithm indexes the similarity of a pair of points by computing the shortest path between them. In the future, it will be interesting to explore whether REMERGE may offer a neurally inspired mechanism by which to learn compact representations of environmental structures.

The mechanism of hippocampal generalization we propose, however, also has important limitations of its own, suggesting it acts in concert with a slow learning neocortical system to support efficient semantic learning and semantic task performance.

1. The hippocampal system may support generalization over relatively short timescales. However, limitations on hippocampal storage capacity, due to its limited size, as well as a wealth of empirical evidence from patients with amnesia and semantic dementia, suggest that semantic knowledge is ultimately consolidated to the neocortex (McClelland et al., 1995; McClelland & Rogers, 2003; Rogers & McClelland, 2004; Scoville & Milner, 1957; Squire, 1992; Squire et al., 1984).

2. Our account emphasizes the principle that individual experiences are represented in the hippocampal system as arbitrary combinations of recurring components (or features), for example, denoting familiar objects, people, odors, or places (Cohen & Eichenbaum, 1993; McClelland & Goddard, 1996; McClelland et al., 1995). We view the neocortex as critical to the gradual development of neural representations that are shaped by the organizational structure of the environment and capture the general characteristics of our experiences (Rogers & McClelland, 2004). One very important characteristic of neocortical learning in this context is the potential for the gradual discovery of useful features that can then be used in rapid hippocampal learning. For example, consider learning a new word—for example, bardrel meaning (say) a kind of bird that lives in Patagonia. If this word is represented in the input to the hippocampus in terms of the specific line segments that make it up, and then a hippocampal representation is formed conjoining the elements of this visual pattern, the basis for generalization to other items will be limited to transformations that preserve the visual feature pattern. However, if this word is represented in the input to the hippocampus as a particular configuration of abstract letters, it will be possible to recognize the same word in distinct versions of the alphabet, including uppercase BARDREL or handwritten script. The development of a system of connection weights that assigns distinct visual patterns (such as a, a, and A) the same internal representation (corresponding to the letter A regardless of the particular graphical depiction) is a crucial function for an intelligent learning system and is thought to arise through gradual neocortical learning (e.g., McClelland & Goddard, 1996). Of note, such learning can assign representations to items corresponding to positions in a multi-dimensional space based only on their relationship to other items. These relationships may then be very abstract conceptual relationships (e.g., father-of, husband-of; Hinton et al., 1986). Just such a set of relationships is likely to be the basis on which the neocortex would learn the abstract correspondences among graphical variants of different letters.

3. It is important to bear in mind that generalization arises through very different mechanisms in a recurrent hippocampal system, compared to network models of the neocortical system. Specifically, the similarity structure of the environment is not statically represented in the internal representations of the network we describe, as it is in networks used to simulate neocortical learning (Rogers & McClelland, 2004), but is instead created on the fly through recurrence. While this property can be useful in affording considerable flexibility in response to changing environments (see below), the representation of regularities present in relatively stationary environments (e.g., familiar concepts, letter forms, objects, etc.) would appear to be most efficiently captured by the stable neural codes viewed to operate in the neocortex. Further, the variability of network output as a function of the level of inhibition, regulated by network temperature, a free parameter in the model, raises the question of how this parameter is controlled, an important issue that deserves consideration in future work.

One scenario that pinpoints a limitation of hippocampal generalization is where successful performance depends on exploiting the shared relational structure between two domains that are perceptually unrelated, as is the case in certain analogical reasoning tasks. The proposed hippocampal scheme would be expected to fail in this context because it lacks any mechanism that would allow transfer from one domain to another, assuming the absence of any cross-domain similarity in the inputs. Recent work has demonstrated that slower-learning models of the neocortex perform well in these settings, for example appropriately generalizing information learnt about one family to another family with an isomorphic relational structure (but no overlap in the inputs and outputs), through learning hidden layer representations that capture the shared structure across domains in the data set (Flusberg, Thibodeau, Sternberg, & Glick, 2011; Hinton et al., 1986; McClelland & Rogers, 2003; Rogers & McClelland, 2008). Such representations could then be exploited for new learning via the hippocampal system that could potentially generalize across domains. More work exploring how hippocampal and neocortical learning systems may work together in such settings is needed.

4. It has recently been shown in an exciting series of experimental studies in rodents that new information that is highly consistent with established neocortical knowledge structures can be rapidly integrated into cortical networks, requiring hippocampal involvement for only a short period (Tse et al., 2007, 2011). These recent findings indicate yet another way in which structured knowledge represented in the neocortex can complement the role of the hippocampus, and they underscore the importance of consistency with existing knowledge structures for allowing rapid neocortical integration. (Although these articles suggest that these new findings might require modification of the complementary learning systems theory, simulations to be reported elsewhere by McClelland, 2011, now demonstrate that in fact such findings are consistent with the characteristics of the existing implementation used to illustrate many of the features of the neocortical learning system, namely that used by Rumelhart & Todd, 1993, and Rogers & McClelland, 2004.)

Taken together, we suggest that the inherent advantages (fast learning, flexibility, facilitates interleaved training) and limitations (dependence on pre-established components and structural relations, variable generalization as a function of inhibition, limited storage capacity) of a recurrence-based hippocampal scheme are largely mirrored by inverse advantages (gradually learnable componential codes that can capture structured relational abstractions, reliable generalization, large storage capacity) and disadvantages (slow learning, requires interleaved training) of neocortical networks. Overall, new developments continue to enrich our understanding of the ways in which complementary action of these two neural systems supports learning and memory, including generalization of what has been learned from new information.

Comment on Constructive Memory: Imagination, Future Thinking

We conclude with a speculation on how the perspective we advocate may have wider implications concerning the contribution of the hippocampus to cognition. A resurgent theme, first popularized by Barlett in the 1930s (Bartlett, 1932), is that memory is constructive in nature, rather than a literal record of the past (Hassabis & Maguire, 2007; McClelland, 2011; Nystrom & McClelland, 1992; Schacter & Addis, 2007). Despite rendering a memory system more prone to errors, it is this constructive aspect of memory that may be its most adaptive, facilitating the creation of new imagined scenarios through the recombination of multiple past experiences, thereby affording a capacity for the simulation of plausible future events. Interestingly, the hippocampus would appear to be critical to this form of generalization, based on evidence that the imagined creations of patients with amnesia are impoverished compared to control subjects (Hassabis, Kumaran, Vann, & Maguire, 2007). The mechanism by which the hippocampus achieves this function—and whether this reflects a specific role in spatial processing (Hassabis & Maguire, 2007; O'Keefe & Nadel, 1978) or a broader role in relational memory binding (Cohen & Eichenbaum, 1993)—remains unclear.

Here, we suggest that the incorporation of recurrence in the computations performed by the hippocampal system naturally affords constructive properties. Interestingly, previous work has shown that the tendency of subjects to show blending errors at the stage of memory recall (i.e., recombination of different sentences in a completion task) can be captured through the operation of a stochastic interactive process similar to the recurrent mechanism implemented in the REMERGE model (Nystrom & McClelland, 1992). Further, introspection on our own imagined creations, and even our dreams, would seem to support a simple intuition: Novel experiences typically involve familiar components (e.g., objects, people), or environmental regularities, rather than entirely fictitious items. Our suggestion, then, is that hippocampal recurrency, in combination with the compositional structure of memories implemented in our model and outlined in other viewpoints (Cohen & Eichenbaum, 1993; McClelland & Goddard, 1996; also see Hummel & Holyoak, 2003), is critical to supporting the arbitrary recombination of elements taken from multiple related episodes that underpins imagination, dreaming, and constructive memory more generally. Indeed, a recurrent memory system, including conjunctive representations that link co-occurring elements composing specific events, combined with top-down inputs from the prefrontal cortex, and the potential for the dynamic variation of network output determined by the level of inhibitory influences, would appear to offer considerable power and flexibility of the kind that may be important for imagination and future thinking.

Future Directions

We have aimed to capture key functional properties of the hippocampal system as a whole, following a simplified scheme designed to reveal how generalization may emerge through a principle of recurrent similarity computation. In future work it will be important to enhance the biological and functional realism of the model by including dynamic learning algorithms and intrinsic variability, and to capture the distinct contributions of anatomically defined brain structures within the hippocampal system, building on earlier extensions of the complementary learning systems framework (McClelland & Goddard, 1996; Norman & O'Reilly, 2003; O'Reilly & Rudy, 2001). These extensions to the model would also allow one to assess whether the core principle we propose, namely recurrent similarity computation, has broader applicability beyond the relatively simple network architectures and experimental scenarios explored here, to generalization in more complex environments.

To achieve this, it will be necessary to demonstrate that desirable functional properties of the REMERGE model (e.g., generalization capacity) can be retained in the context of learned sparse distributed representations of the nature presumed to exist within the hippocampus, an important challenge that remains to be fully addressed by the wider field. Here, we outline some of the key challenges for this endeavor.

In the current formulation, we examined the effects of a highly idealized version of learning on the network's capacity for generalization. Learning was simulated by stipulating the creation of new units for each premise item, then simply varying the relevant connection weights across different runs of the network to capture different levels of learning. Future work will be needed to employ dynamic simulations of a more realistic learning process to explore several issues, including the ability of the network to learn representations of the premise items themselves and the influence of the operation of recurrency during learning on representations in the network. This work should also assess the tendency of the network to learn representations of stored generalizations (e.g., linking different faces paired with the same scene in different episodes) that arise on the feature layer arising through recurrency—and the effect of these processes on the capacity of the network to generalize and yet retain the ability to recall specific experiences.

Here, we have used localist representations in the conjunctive layer, designed to reflect a theoretical extreme idealization of pattern separation in the hippocampus. In fact, however, our theory holds that the hippocampus supports neural representations that are sparser than in other brain regions (e.g., the entorhinal cortex), but that are nevertheless still distributed in nature. In the future, it will be crucial to the further development of the theory to simulate this property in the model, with neural codes in the hidden layer rendered sparser than those in the feature layer through principles outlined in prior formulations—that is, competitive dynamics based on conjunctive coding, inhibitory influences, and low activity levels (e.g., Norman & O'Reilly, 2003; O'Reilly & McClelland, 1994; O'Reilly & Rudy, 2001).

As part of the current work, we have provided initial evidence that recurrency in the hippocampal circuit does not have inevitable costs for recognition memory. In future work, it will be important to examine this issue in more detail—in the context of a recurrent hippocampal system, instantiated using distributed neural codes and incorporating intrinsic variability in processing (i.e., noise). In particular, it would be illuminating to examine the characteristics of the recall signal triggered when study and lure items are presented to the recurrent network—and to compare these distributions to the performance of a feedforward-only hippocampal model (Norman and O'Reilly, 2003) and empirical data concerning the contribution of the hippocampus to recognition memory (e.g., Fortin, Wright, & Eichenbaum, 2004; Wais, Wixted, Hopkins, Squire, 2006). A further important development for this work will be to understand how biologically realistic mechanisms of learning can capture more fully the characteristics of cognitive-level differentiation models of recognition memory (Criss & McClelland, 2006; McClelland & Chappell, 1998; Shiffrin & Steyvers, 1997).

We also hope to explore the possibility that a hippocampal system endowed with recurrent capabilities may facilitate the discovery of environmental structure and its representation in the neocortex, through the replay of generalized activity patterns during offline periods such as sleep. Further research is clearly needed to more fully explore this hypothesis: In broad terms, it would be interesting to assess whether generalized replay arising from a recurrent hippocampal system enhances the efficiency of neocortical learning, compared to the replay of individual episodes as envisioned in the original formulation of complementary learning systems theory.

More generally, we wish to highlight the need to develop a fuller and more systematic treatment of the range of different mechanisms that contribute to what we see as the cardinal aspect of knowledge acquisition, the ability to extract and exploit the general structure of a set of related experiences. Recurrent similarity-based generalization as envisioned here, together with more gradual structure learning as originally envisioned in the complementary learning systems theory, may also co-exist with other forms of semantic learning, including the use of explicit reasoning strategies and the formation of explicitly structured representations such as linear orderings. This understanding should also explore ways in which the processes that lead to these generalizations would still allow retrieval of the particulars of individual events and experiences. The research we present here will hopefully contribute, in part, to the development of such a systematic understanding.

Conclusion

In this article, we have drawn attention to a fundamental, but neglected, tension between theories emphasizing the role of the hippocampus in pattern separation and episodic memory (Marr, 1971; McClelland et al., 1995) and those highlighting its contribution to generalization, often known as flexible or relational memory (Cohen & Eichenbaum, 1993). We have proposed a means by which this apparent conflict may be resolved, through the exploitation of recurrent similarity-based computation in the hippocampal system. This proposal allows a theoretical ideal of pattern separated representations to be retained, while also allowing for efficient generalization. The explanatory value of the REMERGE model will surely rest on future empirical and theoretical developments—at a minimum, we hope our perspective will stimulate discussion and will provoke further investigation of issues that speak to the essential characteristics of learning and memory in the brain.