Developmental Trends and Individual Differences in Brain Systems Involved in Intertemporal Choice During Adolescence

Valuation

One of the neural systems involved in intertemporal choice computes the subjective value of potential rewards to an individual; that is, the value based on personal preferences and contextual experiences, rather than based on some objective measure such as specific monetary value. The mesolimbic dopamine system, including the ventral tegmental area (VTA), ventral striatum (VS), along with the ventromedial prefrontal cortex (vmPFC) have been implicated in this process (reviewed in Ballard & Knutson, 2009; Kable & Glimcher, 2007; Rangel, Camerer & Montague, 2008; Rangel & Hare, 2010).

Of note, there is increasing evidence of both behavioral and neural changes in this valuation system during adolescence (see Casey, Jones, & Hare, 2008, for review). Behaviorally, there are changes under conditions of risky decision making (Crone & van der Molen, 2004) that appear to follow an inverted U-shaped function with maximal sensitivity to reward at midadolescence (Cauffman, Shulman, Steinberg, Claus, Banich, et al., 2010). Concomitant changes are observed in neural systems that process rewards with increased activity in the dopaminergic system, most notably in the ventral striatum, both when anticipating rewards (e.g., Galván, Hare, Parra, Penn, Voss, et al., 2006) and when receiving them (Ernst et al., 2005; van Leijenhorst et al., 2010). As a result, we might expect that intertemporal choice behavior in adolescence will be influenced by changes in neural systems supporting valuation.

Cognitive Control

Brain regions involved in cognitive control, most notably dorsolateral prefrontal cortex (DLPFC), are thought to exert a top-down bias in decision-making tasks by incorporating higher-level goals that are not represented by the valuation system (Peters & Büchel, 2011). Support for this idea comes from neuroimaging studies (Berns et al., 2007; Hare, Camerer, & Rangel, 2009; McClure, Laibson, Loewenstein, & Cohen, 2004) and studies in which brain activity is modulated directly, as by repetitive transcranial magnetic stimulation (rTMS; Figner et al., 2010) or continuous theta burst stimulation (cTBS; Cho et al., 2010). Such top-down biasing by the DLPFC may be especially important under certain conditions. For example, when two options are perceived similarly by the valuation system, additional top-down biasing may be required to select between those two choices (Figner et al., 2010). In other cases, the DLPFC is engaged when the value of rewards are known to be good but are difficult to choose, such as temporally distant offers or healthy but tasteless foods (Hare, Malmaud, & Rangel, 2011).

The neural mechanisms supporting cognitive control are undergoing important changes during adolescence (for review, see Crone, 2009; e.g., Adleman et al., 2002; Rubia et al., 2006). Moreover, such activation has been linked to self-reported control on real-world measures of cognitive and social control (e.g., resistance to peer pressure; Andrews-Hanna et al., 2011). As such, developmental changes during adolescence in cognitive control regions are also likely to play a role in tasks involving intertemporal choice.

Future Thought and Orientation

The third system highlighted by Peters and Büchel (2011) is one involved in imagining the future or future outcomes. This imagery/prospection system relies predominantly on structures within the medial temporal lobe, and regions to which they are anatomically connected, including the retrosplenial cortex, posterior cingulate cortex, lateral parietal cortex, and the vmPFC (e.g., reviewed in Andrews-Hanna, 2012; Schacter, Addis & Buckner, 2008; Spreng, Mar, & Kim, 2009). These cortical regions are also reliably recruited when individuals remember their past (Spreng et al., 2009; Svoboda, McKinnon, & Levine, 2006), suggesting that episodic memory retrieval is an important precursor to prospective thought (Schacter et al., 2008). Prospective processes may be important for intertemporal choice because they allow people to forecast and value events that are not currently affecting them but may come into play at a later time (Benoit, Gilbert, & Burgess, 2011; Cheung & Cardinal, 2005; Gilbert & Wilson, 2007; Peters & Büchel, 2010; Suddendorf, Addis, & Corballis, 2009).

With regards to adolescent development, there is a relative paucity of literature examining changes in the imagery/prospection system during this time period. However, there is reason to think that development of this system may be continuing into adolescence, because a recent behavioral study suggests that self-reported future orientation predicts delay discounting in adolescents better than self-reported impulsivity (Steinberg et al., 2009).

Neurodevelopmental Studies of Delay Discounting

To our knowledge, there is only one study to date that has examined the neural systems involved in intertemporal choice in typically developing adolescents. Christakou, Brammer, and Rubia (2011) gave a standard delay-discounting task in which choices were adjusted based on prior answers to 40 males ranging in age from 12 to 32. They found that adolescents (under the age of 18, N = 19) discounted future rewards more than adults (18 or above, N = 21). Across both the immediate and delayed responses, Christakou and colleagues observed linear effects of age on activation in a number of cognitive control regions spanning prefrontal and parietal cortex, along with decreases in limbic corticostriatal areas. Left vmPFC was the only area in which age-related changes were also associated with individual differences in how rewards were valued. In addition, with increasing age, there was increased connectivity between control regions and the vmPFC, and between the vmPFC and ventral striatum. Hence, this study suggests that the effects of the control system increase with age, those of value system decrease, and that the corticolimbic circuitry becomes more integrated to influence intertemporal choice.

Rationale and Design

Our study was designed to expand upon prior work in a variety of ways. First, compared with the study by Christakou et al. (2011), the present study focused on neural systems underlying intertemporal choice in a more tightly focused age range, individuals between 14 and 19, because behavioral work suggests this is a critical time period for developmental changes in behavior regarding intertemporal choice (Steinberg et al., 2009). Second, it examined the developmental trajectories of the neural systems involved in intertemporal choice—cognitive control, valuation, prospection/imagery. Given that behavioral evidence suggests that both cognitive control and valuation affect intertemporal choice in this age range (Steinberg et al., 2009), we expected to see age-related changes in the brain regions related to these processes. If observed, it would provide confirmatory evidence that the processes influencing intertemporal choice during adolescence are multifaceted. Because the role of prospection/imagery in intertemporal choice in this age range is less clear, our examination of activity in related neural systems was more exploratory.

In addition, we examined how individual differences in factors unrelated to age influence patterns of brain activation. More specifically, the study explored how individual differences in a self-reported composite measure of nonimmediate thinking (NIT), which incorporated both self-reported future orientation and self-reported impulsivity, was related to patterns of brain activation. We predicted that patterns of neural activation would be influenced by individual differences in NIT.

Participants

Advertisements for study participation were placed in a variety of locations throughout metropolitan Denver, including local newspapers, e-mail lists, community colleges, grocery stores, churches, local bus routes, afterschool programs, and bulletin boards at hospitals. Participants were right-handed, spoke English as a native language, and had no history of psychiatric or neurological disorders, substance abuse, head trauma, or claustrophobia. All participants of age provide informed consent, whereas the legal guardians or parents of all participants below the age of 18 provided consent on their behalf. In addition, all minors signed an additional “assent” form explaining the procedures of the study. All procedures were approved by the Colorado Multiple Institutional Review Board and participants were compensated monetarily.

Though the wider sample included participants aged 14–25 years, the present study focused primarily on individuals within the adolescent time period: aged 14–19 years. A total of 37 adolescents met the above criteria and participated in the study. However, six either failed to complete the study or produced unusable imaging data due to scanner artifacts and/or excessive movement (>2 mm linear displacement) and were excluded from analyses. In addition, two participants were excluded based on restricted behavioral performance (i.e., insufficient number of trials in critical conditions). No participants were considered outliers on overall response time (reaction time [RT]).

After eliminating participants based on the above criteria, 29 adolescents yielded usable imaging and behavioral data and their data were included in the analyses described in the present report. In addition to using age as a covariate, we also divided the group in half and compared the older half of the sample (aged 17–19; N = 14) to the younger half (aged 14–16; N = 15). Demographic characteristics of these two age groups are provided in Table 1. Independent-samples t tests indicated that the two age groups did not differ on any demographic characteristics other than age (t = 8.38, df = 25, p < .0001, d = 1.35).

Materials

Procedure

Individuals performed the intertemporal choice task and answered questionnaires as part of a large multipart study that consisted of two sessions of testing on separate days. First, if over 18, informed consent was provided by the participant. If under 18, consent was obtained from the participant’s parent/legal guardian and verbal and written assent was obtained from the participant. Next participants performed the intertemporal choice task described here followed by a Stroop task, the results of which are reported elsewhere (Andrews-Hanna et al., 2011). Afterward participants filled out the self-report questionnaires described below. On the second day, individuals performed another decision-making task in the magnet, after which they left the magnet and the WASI was administered.

Behavioral Data

We first examined whether there were age differences in the proportion of Later compared to Now choices, calculated as follows for each individual: (Number of Later choices – Now choices)/(Total number of choices for that individual). Whereas Now and Later choices were chosen equally often for younger adolescents (aged 14–16, N = 15; mean proportion = .15; one-sample t test: t(14) = 1.14, p = 0.29, d = 2.72), older adolescents (aged 17–19, N = 14) made significantly more Later choices than Now choices (mean proportion = .48; one-sample t test: t(13) = 5.036, p < .001, d = 1.35), suggesting a higher propensity to chose the larger, delayed reward instead of an immediate reward. It is important to note that a statistical comparison between the two groups revealed that older adolescents chose the Later option significantly more than the younger adolescents, t(27) = 2.07, p < .05, d = 0.77 (Table 1). Although the bias in the older group toward a Later choice is somewhat atypical, the bias toward Later choices was greater for the Older than Younger group, consistent with less steep discounting of delayed reward in older adolescents (Steinberg et al., 2009). Despite these categorical group differences, a linear relationship with age was not observed, r(27) = 0.21, r² = 0.04, p = .16.

To examine the relative speed with which each decision was made, we examined response times for Now and Later choices by conducting a repeated-measures analysis of variance (ANOVA) on RTs specifying a within-subjects factor of Choice (Now, Later) and a between-subjects factor of Group (Younger, Older). While there was no significant main effect of Group, F(1, 27) = .002, p = .96, ${\mathrm{\eta }}_{\mathrm{p}}^2=.00$, a Main Effect of Choice was observed, F(1, 27) = 13.09, p < .001, ${\mathrm{\eta }}_{\mathrm{p}}^2=.33$, with responses to Later trials being significantly faster (1.61 s) than to Now trials (1.75 s). It is important to note that this effect was moderated by a significant Group by Choice interaction, F(1, 27) = 5.76, p < .025, ${\mathrm{\eta }}_{\mathrm{p}}^2=.18$. This interaction occurred because whereas the Younger group did not exhibit a significant difference in RT between Now and Later choices (Now: 1.70 s, Later: 1.66 s, Paired-samples t test), t(14) = 1.03, p > .32, d = 0.23, the Older group was significantly faster for Later (1.60 s) than Now (1.80 s) choices; paired-samples t test: t(13) = 3.67, p < .01, d = 1.00. Paralleling this finding, the difference in RTs between Now and Later choices correlated significantly with age, r(27) = 0.40, r² = 0.16; p < .05.

Although RT is not necessarily a pure measure of decision difficulty because it may reflect other factors (e.g., lapses in attention; see de Wit, 2009), this pattern of findings is consistent with the idea that decisions about Now trials require more deliberation than Later trials for older adolescents. For example, increases in performance during adolescence on the Tower of London task, which requires the planning of a multistep solution, appears to be mediated by increases in the time it takes to initiate the first move (Albert & Steinberg, 2011), which likely reflects increasing deliberation before acting. Moreover, the longer the latency to first move, the lower the degree of self-reported impulsivity as measured by the Barratt Impulsiveness Scale, Version 11. Consistent with this interpretation that increased RT reflects increased deliberation, the difference in RT between Now and Later responses strongly correlated with the proportion of Later responses, r(27) = 0.75, r² = 0.56, p < .0001, even when controlling for the effect of age on both variables, r(27) = 0.73, r² = 0.53, p < .0001. In other words, individuals with a greater propensity to choose the delayed reward responded more slowly for Now choices compared with Later choices relative to those individuals with a greater propensity to choose the immediate reward. The sum of the behavioral results are consistent with the view that older adolescents are more deliberative in their decision-making compared with younger adolescence because they tended to be more biased toward the delayed response and also differentiated between immediate and delayed responses to a greater degree.

Our next question was whether such evidence of future-oriented and deliberate thought extended outside the laboratory to measures of behavior in everyday life. To do so, we investigated whether there were group differences on the composite measure NIT. This measure yielded a slight trend toward greater NIT for the older (M = 0.27) than younger group (M = −0.26), t(27) = 1.16, p = .12, d = .56, and no linear correlation with age (Table 1). It is interesting to note that individuals who self-reported more NIT trended toward a higher proportions of Later trials compared with Now trials, r(26) = 0.35, r² = 0.12, p = .065, and slower responses on Now trials compared to Later trials, r(26) = 0.36, r² = 0.13, p = .061, when controlling for the effect of age on both variables. This pattern of associations suggests that behavioral responding on our version of the delay-discounting task has at least some relationship to more real-world measures of underlying constructs related to intertemporal choice.

Collectively, the laboratory tasks and self-report measures converge to suggest that adolescence is an important developmental time period marked by increased tendencies toward more future-oriented and deliberate thought. However, despite these important developmental relationships, the analysis of NIT points to significant individual variability among such behaviors that cannot be solely attributed to age. The goal of the remaining analyses was to examine the neurobiological sources of this age-related and non-age related variability in intertemporal choice.

fMRI Data

Relationships of Age and NIT to Control, Valuation, and Prospection Systems

The results of the current study suggest important developmental changes in all three of the neural systems posited by Peters and Büchel (2011) to play a role in intertemporal choice—control, valuation, and prospection. Perhaps the most striking finding was that activity across brain regions implicated in aiding in intertemporal choice became more differentiated for Now versus Later choices with increasing age during adolescence. As such, it suggests a developmental trajectory during adolescence in which thinking about the relative merits of rewards in the present versus those in the future may start to be treated differently. While not directly in conflict with models suggesting that cortical control regions slowly gain control over highly active reward systems during adolescence (e.g., Casey & Jones, 2010), the current findings suggest a somewhat more complicated picture. Of note, it highlights the contribution of the prospection system, which has been hypothesized to be important for projecting the future value of reward and actions. Currently, the development of this system during adolescence has been relatively unexplored, and hence seems ripe for future investigation.

Because behavior varied across age, the study also examined which of the age-related changes remained when such behavioral differences were considered statistically. Although one cannot truly control for these differences (Miller & Chapman, 2001), and indeed, they may indeed represent important developmental effects, it is nonetheless instructive to consider the pattern of results. Notably, what age differences remained after taking into account differences in behavioral performance were generally related to the cognitive control system, and mainly, but not exclusively, involved frontal regions (Table 3, righthand side). These findings highlight the ongoing development and engagement of prefrontal regions in decision making during adolescence and are consistent with our prior results indicating increased engagement of these regions during adolescence for cognitive control over nonaffective stimuli (Andrews-Hanna et al., 2011). They are also consistent with our prior behavioral work suggesting that nonaffective factors play an important role in the ability to delay reward during adolescence (Steinberg et al., 2009).

Also of importance was the finding that patterns of brain activity were related to individual differences in a composite measure of NIT (that indexed the joint variance between Impulse Control and Future Orientation) independent of age. In general, the pattern of results suggests that the more a youth engages in NIT, the less there was a need to engage cognitive control in making decisions regarding current or future rewards. Although this result may seem counterintuitive, in that one might expect those adolescents with greater rather than less NIT to have engaged cognitive control regions to a higher degree, this finding is consistent research indicating that superior performance in nonclinical groups can be associated with reduced rather than increased brain activation (e.g., Burgess et al., 2010; Sheridan, Hinshaw & D’Esposito, 2007). There were also hints of potential involvement of the prospection system, as increased NIT was associated with increased activity in the parahippocampal region bilaterally, although the level of activity for these regions did not reach significance levels required for cluster correction.

Of note, the regions isolated in the NIT analysis are nonoverlapping with those identified in the age analysis. For example, the parahippocampal region that was identified as being sensitive to age effects is more posterior and discontinuous with that identified as associated with NIT. Statistically, it is possible that activity in a given region could be influenced by both factors simultaneously, but independently. Yet in the current study the set of regions whose activity is associated with age appears to be separable from those influenced by NIT. As such, the pattern that emerges from the present study is one suggesting that a complicated interaction between developmental effects and individual differences in NIT influence delay-discounting behavior during adolescence.

The results of the current study stand somewhat in contrast to Christakou et al. (2011). In their sample of 40 males ranging in age from 12 to 32, they found that increasing age was associated with linear increases in activity in portions of the DLPFC, vmPFC, parietal and superior temporal regions along with linear decreases in activation in the ventral striatum, insula, anterior cingulate, occipital and other portions of parietal cortex. In addition, they found that with increasing age there was increased functional coactivation of vmPFC with the DLFPC for delayed reward and with the ventral striatum for immediate reward. They interpreted this pattern as suggesting that maturation of fronto-striatal circuitry is associated with increase in the ability to delay reward with age. They also found that the only region of the brain in which increasing activity with age was also associated with behavioral changes, as indexed by reduced discounting of future rewards, was a portion of the left vmPFC during immediate reward. As such, they suggested that this region as well as its connectivity to other brain regions may play an important role in maturational and behavioral changes in delay discounting.

Our results are consistent with theirs in suggesting increased activation of prefrontal cognitive control regions with age. Although we did not find any effects of age for the ventral striatum, we did find activation in the VTA in the contrast of Now versus Later and of the anterior regions of the right vmPFC for Now versus Fixation, the latter effect which remained even after age differences in behavior were considered. Thus, our results are consistent with theirs in suggesting age-related changes in cognitive control regions as well as those related to valuation. In addition, our study additionally suggests that age-related changes in the prospection system may also influence intertemporal choice.

There are a number of potential reasons for the differences in results between their study and ours. First, their sample included a much large age range, from 12 to 40, whereas our focused on development during the narrower age period, that of 14 to 19. Second, their study was limited to males, whereas ours included both males and females. Third, we used a different method than they did to assess behaviors related to intertemporal choice. Finally, we used a stringent cluster-wise correction for consideration of statistically significant results. For example the region of vmPFC that they found sensitive to both age and behavioral effects consisted of just 8 voxels, and many of the regions that they discuss in their results consist of clusters of well under 100 voxels. Hence, differences in the criteria used to deem regional brain activation as being significant may have contributed to the dissimilarities observed between their study and ours.

Implications for Substance Abuse

A heightened bias to select an immediate compared with a delayed reward has been linked to the predisposition, acquisition, and maintenance of substance abuse. For example, a bias toward immediate rewards is predictive of individuals who are at risk for substance abuse (see Carroll, Anker, Mach, Newman, & Perry, 2010 for review), such as individuals with a family history of substance and alcohol abuse (Acheson, Vincent, Sorocco, & Lovallo, 2011). Individuals who actually abuse substances also exhibit such a bias (for reviews, see Reynolds, 2006; Yi, Mitchell, & Bickel, 2010). Moreover, a bias toward immediate reward has been recently demonstrated to predict treatment outcome for substance abuse in adolescents (Stanger et al., 2012). Hence, understanding the behavioral and neural systems underlying intertemporal choice is likely to have important implications for identifying individuals at risk for substance abuse and for those who are likely to need more intensive therapeutic interventions once they have become dependent on substances.

Our results, and those in the larger literature, suggest that tasks of intertemporal choice affect a variety of brain systems. As such, there are likely to be multiple factors that can lead to a bias toward choosing an immediate reward over a delayed one. This pattern of findings has important implications for potential treatment because it indicates that impulsive choice for a current reward might result because of (a) an inability to control or inhibit urges or prepotent responses (i.e., deficits in the cognitive control system), (b) an overvaluation of rewards (i.e., alterations in the valuation system, or (c) an inability to conceptualize the future (i.e., alternations in the prospection system).

Our results suggested that activation of cognitive control regions was related to age and also to individual differences in NIT. With regards to age, interventions to preclude the onset of substance abuse during adolescence would be well to consider the age of the targeted group. For example, at younger ages, on average the neural machinery required to “Just Say No” may not yet be mature enough to be effective. In addition to age effects, individual youth may also vary in their ability to invoke such cognitive control, as indicated by the variation in patterns of brain activity related to NIT.

With regards to valuation, we found that cortical regions (e.g., vmPFC) were affected by age but not as clearly by individual differences in NIT. These findings are consistent with the suggestion that impulsive behavior in adolescence is partly due to underdeveloped cortical valuation systems (Galván et al., 2006). In particular, it has been proposed that impulsive behavior in adolescents arises not merely from an overactive reward system in the striatum, but rather because it is coupled with an underdeveloped vmPFC. Thus, interventions that consider how adolescents value rewards versus punishments may be particularly fruitful to pursue.

Finally, the role of the prospection system in substance abuse is relatively unexplored, but may play a role in that the ability imagine the future relies, in part, on being able to access memories from the past that are relevant and related. We found age-related increases in activity in posterior portions of the parahippocampal gyrus and a hint of an influence of NIT on more anterior regions of the parahippocampus bilaterally. To the degree that younger adolescents will have fewer memories than older ones, they may have less of a storehouse of memories on which to base their predictions about the future. These findings also suggest the possibility that environmental experiences may have an important influence with regards to how choices are perceived. Consider a young adolescent experimenting with drugs for the first time. If that experience is positive—good feelings when getting high, acceptance by friends or increase in status, sense of thrill and adventure—that will be encoded by a memory system and could potentially bias valuation toward current reward. Many such experiences might build upon that initial bias and increase it. If on the other hand that initial experience is negative—altered feelings leading to anxiety or a sense of loss of control, punishment by elders or authorities, disconnection from peers—then the system may become biased toward considering future outcomes, which once again, could snowball with increasing numbers of such experiences. Such considerations provide the possibility that an adolescent’s early experiences with substance of abuse may provide a memory base from which the value of using drugs can possibly be enhanced, but may also be reduced. Thus, it suggests that experiences during this developmental period may be particularly important.

Potential Limitations

The current study has certain limitations that should be noted. First, we did not use a typical delay discounting procedure that hones in on an indifference point where the subjective value of an immediate choice is equal to that of a delayed choice. As such, the generalization of our results to studies using this more traditional method of examining delay discounting may be limited. Nonetheless, the areas activated by our version of the task overlap with those reported in a meta-analysis of tasks involving intertemporal choice (Carter et al., 2010) as well as with an empirical study by Christakou et al. (2011) in which the age of half of the participants overlapped with ours.

The method we did employ, one in which the value of immediate choices given to individuals of different age groups varied based on prior research (Steinberg et al., 2009), was intended to be sensitive to the fact that the same amount of money in a delay discounting task appears to have different value for individuals of different ages. However, we may have introduced other confounds, such as the possibility that different age groups found the task to be of varying degrees of difficulty. In addition, because of the different range of monetary choices across age groups as well as the somewhat restricted range of those choices, our procedure prohibited us from effectively calculating an individual’s discounting rate with any degree of confidence.

Another potential limitation of the current study is that we did not find the typical bias toward Now choices. Nonetheless, our data did yield evidence for a developmental trend toward a greater proportion of Later choices in older adolescents, consistent with other studies in the literature finding less steep discounting functions with increasing age (Christakou et al., 2011; Steinberg et al., 2009). Moreover, our RT data are consistent with the idea that older adolescents were more thoughtful and deliberative for Now compared with Later choices than were younger adolescents. It should be noted, however, that RT is not a pure measure of difficulty, thoughtfulness or deliberation and might index other processes as well such as attentional lapses. However, if RT does index, at least in part, some aspect of difficulty or deliberation, then the more extensive activation of prefrontal regions involved in cognitive control with increasing age for the contrast of Now > Later trials would not be surprising, as these regions are known to be engaged under conditions of high demand. Our findings are consistent with TMS studies showing that the effect of TMS over DLPFC is most pronounced for difficult intertemporal choices (Figner et al., 2010). From that perspective, the pattern of results in our study is instructive. It suggests that control mechanisms are invoked whenever intertemporal choices are difficult, and their role is not necessarily restricted to conditions in which the choice for a delayed response is selected.

In addition, there may be limitations to some of the measures used. For example, in the present study we used hypothetical choices instead of realized ones. While hypothetical and realized choices have been shown to yield very similar results in adults, this issue has not been directly tested in adolescents. In addition, our measure of impulsivity was a self-report measure that, although having good psychometric properties, is limited.

Finally, in the present study we employed a cross-sectional design, which does not provide direct evidence of change that occurs with development. Furthermore, work with longitudinal investigations will be required to determine whether the patterns we observed characterize developmental trajectories over time.