Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Nov 5.
Published in final edited form as: Vis cogn. 2009 Apr 1;18(2):10.1080/13506280902734326. doi: 10.1080/13506280902734326

Why is Information Displaced from Visual Working Memory during Visual Search?

Geoffrey F Woodman 1, Steven J Luck 2
PMCID: PMC3817820  NIHMSID: NIHMS523598  PMID: 24204180

Abstract

Research has shown that performing visual search while maintaining representations in visual working memory displaces up to one object’s worth of information from memory. This memory displacement has previously been attributed to a nonspecific disruption of the memory representation by the mere presentation of the visual search array, and the goal of the present study was to determine whether it instead reflects the use of visual working memory in the actual search process. The first hypothesis tested was that working memory displacement occurs because observers preemptively discard about an object’s worth of information from visual working memory in anticipation of performing visual search. Second, we tested the hypothesis that on target-absent trials no information is displaced from visual working memory because no target is entered into memory when search is completed. Finally, we tested whether visual working memory displacement is due to the need to select a response to the search array. The findings rule out these alternative explanations. The present study supports the hypothesis that change-detection performance is impaired when a search array appears during the retention interval due to nonspecific disruption or masking.

Models of cognition and attention propose that visual working memory plays a pivotal role in many tasks and cognitive operations. For example, models of visual search have proposed that visual working memory is used to store a template of the target (e.g., Desimone & Duncan, 1995), categorize the objects in the search array (e.g., Bundesen, 1990), and compare the target representation to possible target objects (e.g., Duncan & Humphreys, 1989). These central roles for visual working memory in search have been drawn into question by empirical findings that the efficiency of visual search is not impaired by simultaneously maintaining objects in visual working memory in service of a change-detection task (Oh & Kim, 2004; Woodman, 2002; Woodman, Vogel, & Luck, 2001). This previous work found that, although search efficiency was unchanged during concurrent task performance, some information was displaced from visual working memory when search was performed. The present study tests three hypotheses regarding the cause of the loss of information from visual working memory. First, we tested the hypothesis that information is expelled from visual working memory in anticipation of the need to perform visual search. Second, we tested whether information is displaced from visual working memory by the target entering this limited-capacity memory store. Third, we tested the hypothesis that information is displaced from visual working memory due to the need to perform response selection during visual search. Thus, the present study serves to further characterize how visual working memory representations influence the perception of new information and visually guided behavior.

Numerous models of visual search have proposed that visual working memory is critical for processing objects in complex visual arrays. One set of models propose that a target template is stored in visual working memory during visual search (Bundesen, 1990; Desimone & Duncan, 1995; Duncan & Humphreys, 1989)1. This template representation is hypothesized to bias neurons in visual cortex to preferentially process objects in the visual field with similar features. In addition, some of these models also propose that when possible target objects are selected by attention that they need to be encoded into visual working memory to be compared with the target representation (Bundesen, 1990; Duncan & Humphreys, 1989). Given the pivotal roles that visual working memory is proposed to play in models of visual search, it is necessary to empirically test whether visual search depends upon visual working memory. Woodman, Vogel, and Luck (2001) previously tested such models using dual-task experiments that required participants to perform visual search during the retention interval of a change-detection task.

Using a dual-task paradigm introduced decades before (i.e., Logan, 1978), Woodman et al. (2001) studied the nature of the interactions between visual working memory and attentional selection during search by requiring observers to perform visual search during the retention interval of a change-detection task. On dual-task trials, observers were first shown a sample array of objects to remember for 500 ms followed by a 500 ms blank interval. Then, a visual search array was presented for 4000 ms followed by another 500 ms blank interval; the observers made a speeded response to the search array. Finally, a memory test array was presented for 2000 ms, and the observers reported whether the sample and test arrays were identical. Single-task conditions were also tested (memory alone and search alone), in which the stimulus arrays for the other task were replaced with blank intervals. The search elements were different spatial configurations of the same line segments that formed Landolt-C-like stimuli. During the search task observers were required to find one of these objects with a gap on its top or bottom and press the appropriate button to indicate the identity of the target on that trial. At the same time participants were required to hold in visual working memory representations of four colored squares (Experiment 1) or four Landolt-C stimuli that were identical to the search elements (Experiment 2). Woodman and colleagues found that the slopes of the search functions were not different whether search was performed during the retention interval of the change-detection task or search was performed in isolation. These findings show that the efficiency of visual search is not impaired by performing visual search when visual working memory is full.

The findings of Woodman et al. (2001) challenge models of the operation of visual attention during visual search, which propose that visual working memory plays several key roles in visual search, as discussed above. However, it is important to consider whether observers simply discarded information from visual working memory so that they could perform the search task efficiently. If so, then these findings would not provide a challenge to theories of search.

In an initial attempt to address this issue, Woodman et al. (2001) examined whether change-detection accuracy declined in the dual-task condition compared to the memory-alone single task condition. They found that performing visual search during the memory retention interval did indeed reduce change detection accuracy relative to the baseline condition in which the retention interval was blank and no search task was performed. Specifically, the amount of information available in working memory at the time of test was reduced by an amount comparable to 0.5–1 objects when search was performed concurrently. This reduction in change-detection performance was approximately the same magnitude regardless of the set size of the search array presented during the retention interval.

Insensitivity to the set size of the search array suggested that the interference was caused by the mere presentation of the search array and did not depend upon the search task itself. To test this hypothesis, Woodman et al. (2001) conducted an additional experiment in which the participants were instructed to completely ignore the search arrays. Passively viewing the visual search arrays lead to approximately the same magnitude drop in change-detection accuracy as performing visual search on these arrays. This reduction in the ability of observers to maintain the object representations in visual working memory was not spatially specific, because the search arrays never spatially overlapped with the change-detection stimuli in any of the experiments. Thus, Woodman et al. (2001) concluded that information was being displaced from visual working memory due to the search arrays having a nonspecific disruption or masking effect on the stored representations of the objects from the memory array. Typically, masking is observed when the masks appear close in space and time to the masked stimulus (Breitmeyer, 1984). The disruption of visual working memory observed in Woodman et al. (2001) was called nonspecific because the search arrays and memory arrays were presented far apart in both space (i.e., at least 0.5 degrees of visual angle) and time (i.e., 500 ms).

However, it is possible that the subjects performed some sort of search on the stimuli in this experiment even though they were instructed to ignore the search arrays. This alternative explanation is based on the fact that the search arrays that were passively viewed were identical to those used in the experiments that required search for an object in the array. Specifically, each array contained a unique object (one square with a gap up or down) in the context of dissimilar distractors (stimuli with gaps left and right). It is possible that subjects noticed that each of the arrays shown during the retention intervals had a unique element embedded in it. That is, as with all passive-viewing conditions, experimenters lose control over what task is performed and subjects essentially choose what task to perform for themselves. As a result, it is possible that having noticed this regularity in every search array, the observers spontaneously started searching for the unique object in the arrays during the 4 seconds that the search array was presented on each trial. Having found the unique object they stored it in visual working memory. Thus, this experiment did not conclusively rule out the possibility that an active search process was responsible for the observed reduction in memory performance.

Because of the substantial theoretical importance of the role of visual working memory in visual search, we sought to test several alternative explanations for the reduction in working memory performance that occurs when a search task is interposed during the delay interval. tested the hypothesis that that information is expelled from working memory in expectation of performing visual search. Experiment 2 tested the hypothesis that the visual search target is stored in visual working memory once it is detected. Experiment 3 tested the hypothesis that it is necessary to store some form of information about the search array in visual working memory to generate a response. Although each of these hypotheses is based on well-respected theories of attention, none of them was supported by the results of the experiments. These results are therefore best explained by the original hypothesis, namely that the presentation of a new stimulus array produces a nonspecific disruption of visual working memory. This increases the strength of the conclusion that the object working memory system is not used for typical visual search tasks.

Experiment 1

The findings of Woodman et al (2001) are consistent with the predictions of models, such as the biased competition account (Desimone & Duncan, 1995), that propose a target representation is entered into visual working memory to bias perceptual attention to target items. Such models are not specific about when the target template is encoded into visual working memory. However, one logical possibility consistent with neurophysiological data (e.g., Chelazzi, Miller, Duncan, & Desimone, 1993) is that this is done in advance of the presentation of the search array to give the template representation the opportunity to influence the appropriate posterior visual cells that are selective for the features of the target. Alternatively, other models propose that some or all of the information in visual working memory can be strategically expelled to provide the representational space necessary for search to be performed (Duncan & Humphreys, 1989). Thus, multiple models are consistent with the prediction that a subset of the information stored in visual working memory would be displaced in anticipation of performing visual search. To test this prediction, we required observers to perform visual search for a target among 3, 7, or 11 distractor objects during the retention interval of a change-detection task. Unpredictably and infrequently, we interleaved trials in which no search array was presented during the retention interval. Change-detection performance on these blank trials provided a measure of whether information was displaced from visual working memory when the observers anticipated performing a search task but no array was presented. If a target template representation is encoded into visual working memory prior to search to bias attention toward similar objects, thus displacing existing information in visual working memory, then we should also observe decreased change-detection accuracy on these blank trials relative to the condition in which only the memory task was performed. In contrast, if information is displaced from visual working memory due to nonspecific masking or disruption of maintenance, then change-detection performance should be similar on the blank trials in the dual-task condition and in the condition in which only the memory task was performed.

Methods

Participants

The participants were 10 undergraduates (age 18–32) from the University of Iowa who all had normal or corrected-to-normal vision. They participated for partial fulfillment of a course requirement after informed consent was obtained.

Stimuli

The stimuli were presented on a homogeneous gray background (9.9 cd/m2) on a PC-controlled CRT. As illustrated in Figure 1, the memory stimuli consisted of four colored squares (each 0.45° × 0.45°) that were centered 0.68°from fixation. The color of each square was selected without replacement from a set of seven colors (red, green, blue, black, white, yellow, and violet, see Vogel, Woodman, & Luck, 2001 for precise color values).

Figure 1.

Figure 1

Open in a new tab

An example of the sequence of stimuli presented during the dual-task condition of Experiment 1. The letters to the left indicate that subjects began each trial by repeating the articulatory-suppression load. There were 500-ms intervals between each stimulus frame in which the screen was blank except for the central fixation point.

The search arrays were composed of 4, 8, or 12 black squares with gaps (each 0.45° × 0.45°, 0.08°line thickness, 0.12°gap width, > 0.01 cd/m2). The nontarget objects had gaps on either the left or right side and the targets had a gap on the top or the bottom of the square. The items were presented in a 6.1° × 6.1°region, beginning at least 1°from fixation, with a minimum center-to-center inter-object distance of 0.6°. To control for possible effects of density differences between set sizes (e.g., Cohen & Ivry, 1991) the items were presented in groups of 4 in each quadrant and the number of filled quadrants was manipulated.

Procedure

Figure 1 depicts an example of the sequence of events that occurred on each trial in the dual-task condition. Following the 500-ms presentation of a central fixation cross the memory array of four colored squares was presented for 500 ms. Then, a 500-ms blank interval preceded the 4000-ms presentation of the search array. Observers were instructed to use their index and middle fingers on their dominant hand to respond as fast as possible to the search array by pressing one of two buttons on a game pad indicating whether the target on each trial had a gap up or down. On 25% of randomly selected dual-task trials, no search array was presented but was replaced with a 4000-ms blank interval. Finally, an additional 500-ms blank interval preceded the presentation of the memory-test array for 2000 ms. The index and middle fingers on their non-dominant hand were used to report whether each memory-test array was identical to the memory-sample array or differed in that one of the colors was replaced with a color not seen in the memory array. For the memory task accuracy was stressed and responses were unspeeded.

In the memory-alone condition, the search array was replaced by a 4000-ms blank interval. Similarly, in the search-alone condition the memory and memory-test arrays were replaced by blank intervals. The three conditions were blocked with 48 trials per set size in the search and dual-task conditions and 48 change-detection trials in the memory-alone condition. Order of conditions was randomized for each participant. Approximately 10 practice trials were given prior to each condition block. At the beginning of each block of trials, observers were shown a set of 4 letters or digits that they were required to repeat aloud during each trial of that block. The white alphanumeric characters (either, “a, b, c, d,” “w, x, y, z,” “1, 2, 3, 4,” or “6, 7, 8, 9”) were presented for 1500 ms, 1500 ms before the beginning of the first trial. Participants were allowed to stop articulating the verbal load at the end of the block of trials at which time they were allowed to take as long a break as they required before continuing.

Analysis

The visual search reaction times (RTs) were entered into an ANOVA with the factors of condition (search-alone versus dual-task search) and set size (4,8, or 12 items). Change-detection performance was analyzed by entering accuracy from each condition and search set size into an ANOVA (memory-alone, set size 4, 8, 12, or no-search array) and then performing planned pair-wise comparisons. We also used change-detection accuracy to compute K values (Cowan, 2001; Pashler, 1988), which serves to estimate the number of object’s worth of information stored in visual working memory across conditions (e.g., Woodman et al., 2001).

Results

Visual search accuracy was high across all trial types and subjects (i.e., above 96.5% correct) and did not significantly differ between conditions or set sizes (F < 1). Mean visual search RTs are shown in Figure 2A. We found that search RT was approximately 311 ms slower in the dual-task condition than in the search-alone condition across all set sizes, but this was due to a difference in y-intercept rather than a difference in slope. Specifically, the slopes of the search functions were very similar at 49.9 ms/item in the dual-task condition and 57.1 ms/item in the search-alone condition. Supporting these observations are the results of the ANOVA that yielded a significant main effect of set size (F(2,18) = 108.68, p > .001); the effect of condition approached significance (F(1,9) = 2.21, p = .17), but no interaction between these factors was found (F < 1.0).

Figure 2.

Figure 2

Open in a new tab

The results from Experiment 1. A: The visual search RTs from the dual-task condition (dashed lines, empty symbols) and the search-alone condition (solid lines, filled symbols). B: The change-detection accuracy in the memory-alone condition, left, and the dual-task condition as a function of set size or the presentation of a blank, right. The error bars in this and all subsequent figures show the 95% within subject’s confidence intervals (Loftus & Loftus, 1988).

Of primary interest in the present study was how the different types of search trials in the dual-task condition influenced change-detection performance. As shown in Figure 2B, observers were less accurate at detecting changes when a search array was presented. This resulted in a significant effect of trial type (F(4,36) = 8.03, p > .001). However, the size of the impairment on the change-detection task was not sensitive to the set size of the interposed search array (for set size 4, 8, and 12 accuracy was 74.4%, 70.3%, and 72.5% correct, respectively) and therefore did not differ significantly between set sizes (all pair-wise ps > .25). On the no-search array trials, change-detection performance was statistically indistinguishable from accuracy in the memory-alone condition (83.4% versus 83.6%, respectively, F < 1.0) and change-detection performance in both of these contexts was significantly better than accuracy on any of the search trials in planned comparisons (all ps < .05). To provide another way to quantify change-detection performance we also calculated K from the mean hit and false-alarm rates. K estimates were very similar in the memory-alone condition (K = 2.70 objects) and on no-search array trials (K = 2.67 objects) but were reduced by approximately one object’s worth of information when search of any set size was performed during the retention interval (K = 1.95, 1.63, and 1.80 for set size 4, 8, and 12 trials, respectively). In summary, change-detection performance was approximately equal during the memory-alone condition and the no-search array trials and performance was superior on both of these trial types compared to when any search array was processed during the retention interval.

Discussion

In Experiment 1 we replicated previous reports that the efficiency of visual search was not impaired by concurrently maintaining objects in visual working memory to perform a change-detection task (Oh & Kim, 2004; Woodman et al., 2001). In addition, we found that when search was performed during the retention interval of the change-detection task approximately one object’s worth of information was displaced from visual working memory. These change-detection results also replicate those reported in Experiment 1 of Woodman et al. (2001). However, of primary interest was change-detection performance on the no-search array trials in the dual-task condition. If information were expelled from visual working memory due to the expectation of performing visual search, then change-detection performance on the no-search array trials would be similar to that on the randomly interleaved search trials. In contrast, we found that change-detection performance on the blank trials in the dual-task condition was equivalent to accuracy in the memory-alone condition. These results support the hypothesis that information is not displaced from visual working memory in anticipation of performing visual search.

Information could have been expelled from visual working memory in preparation for performing visual search for a variety of reasons. Theories of attention propose that a target template representation is encoded into visual working memory in preparation for performing visual search (Desimone & Duncan, 1995). The working memory template then biases perceptual processing mechanisms to select similar perceptual inputs. It has also been suggested that, if search is to be performed and visual working memory is full, then some or all of the contents of this memory store may be purged to provide representational space for the search elements to be encoded into memory (Duncan & Humphreys, 1989). This is necessary in several models because they propose that visual working memory is where representations of items in the search array are categorized and compared to the target template during search (Bundesen, 1990; Duncan & Humphreys, 1989). The findings reported above provide evidence that contradicts these theoretical predictions because they show that information was not displaced from visual working memory in anticipation of performing visual search even when visual working memory is full. Prior evidence for a role of visual working memory in visual search comes primarily from studies in which the target is cued on every trial rather than remaining constant over trials (Chelazzi, Duncan, Miller, & Desimone, 1998; Chelazzi, Miller, Duncan, & Desimone, 1993). In such cases, it may be necessary to store a representation of the target it visual working memory, but this may not be necessary when the target remains constant from trial to trial. Indeed, we have shown that a concurrent visual working memory load interferes with search when the target is cued on each trial but not when it remains constant from trial to trial (Woodman, Luck, & Schall, 2007). The present results provide additional evidence that visual working memory plays little or no role in visual search when the target remains constant over trials.

Although the present findings challenge models of visual search that propose visual working memory target representations bias attention toward target items, these findings may be consistent with a role for visual working memory in storing the target representation once it is found. Thus, the next hypothesis we tested was that information is displaced from visual working memory by the encoding of the target into visual working memory once perceptual attention is focused on it.

Experiment 2

In Experiment 2, we tested the hypothesis that the necessity to encode the visual search target into visual working memory displaced object representations maintained in service of the change-detection task. The experiments reported in Woodman et al. (2001) always used a search paradigm in which a target was present on each trial and the observers’ task was to report which of two possible target shapes was presented. A target was always presented to discourage subjects from adopting a time-out strategy for no-target responses during visual search (Chun & Wolfe, 1996). This aspect of the experimental design leaves open the possibility that the drop in change-detection accuracy observed in the previous experiments was due to the search target representation being encoded into the visual working memory store. In Experiment 2, we tested this hypothesis by requiring observers to perform a target-present versus -absent visual search task in which no target object was present in half of the search arrays. We predicted that if the target were displacing information from visual working memory, then on target-absent trials change-detection performance should be equivalent to change-detection accuracy in the memory-alone condition. However, if information is displaced by the presentation of any array of objects, as predicted by the nonspecific disruption or masking account, then change-detection should be worse on target-absent trials compared to performance in the memory-alone condition.

Method

The methods used in Experiment 2 were identical to those of Experiment 1 with the following exceptions. A new group of 10 observers participated for course credit after informed consent was obtained. The search task in the search alone and dual-task conditions required a target-present versus target-absent response on each trial. Target-present and -absent trials were equally probable and randomly interleaved. The target object was either a square with a gap on the top or on the bottom (counterbalanced across participants) and distractors were squares with gaps left and right. Unlike in Experiment 1, there were no trials in the dual-task condition in which no search array was presented. In sum, the design of the present experiment was identical to Experiment 1 in Woodman et al. (2001) with the exception that targets were present only on half of the trials and the response observers needed to make indicated target presence versus absence on each trial.

Results

Accuracy of the responses to the visual search arrays was above 94.6 % correct across all trials types and did not significantly differ across conditions and set sizes (F < 1). The mean target-present and-absent search RTs are shown in Figure 3A. As is evident from this graph, responses were faster in the search-alone condition (mean = 1547.7 ms) than in the dual-task condition (mean =1733.8 ms) across set sizes and target-present and-absent trials. Performing visual search in the dual-task condition resulted in a y-intercept difference compared to search alone but the slopes of the search functions between conditions were essentially the same. That is, RT increased more rapidly with set size when the target was absent than when it was present, but the slopes of the search functions were similar for search performed alone (target-absent = 146.1 ms/item; target-present = 60.7 ms/item) or during the dual-task condition (target-absent = 131.9 ms/item; target-present = 54.6 ms/item). These patterns of visual search RTs led to significant main effects of condition (F(1,9) = 9.99, p < .05), target presence (F(1,9) = 136.97, p < .001), and set size (F(2,18) = 203.31, p < .001). There was also a significant interaction of set size and target presence (F(1,9) = 34.19, p < .001). No other interactions were significant (ps > .20).

Figure 3.

Figure 3

Open in a new tab

The results from Experiment 2. A: The visual search RTs from the dual-task (dashed lines, empty symbols) and the search-alone condition (solid lines, filled symbols) for target present (circles) and target-absent trials (squares). B: The change-detection accuracy in the memory-alone condition, left, and the dual-task condition as a function of set size and target presence, right.

Change-detection accuracy for each trial type is shown in Figure 3B. Accuracy was highest in the memory-alone condition (mean = 83.0 % correct) and lower when any type of search array was viewed during the retention interval in the dual-task condition (means = 74.1, 77.2, 75.5, 72.1, 70.8, 66.1% correct, for target-present across set size 4, 8, and 12 versus target-absent across the three set sizes, respectively). These observations were confirmed by the results of an ANOVA with the factors of target presence (present versus absent) and set size (4, 8, or 12) from which the main effect of set size approached significance (F(2,18) = 3.06, p = .072) but neither the main effect of target presence nor the interaction were significant (ps > .30). Pairwise planned comparisons of change-detection accuracy in the memory-alone condition compared to change detection during target-absent search trials at each set size of the dual-task condition resulted in a significant differences across target-absent trials of all set sizes (ps < .05). The calculated K value was a half to one object’s worth of information higher in the memory-alone condition (K = 2.64) than when any of the search arrays were viewed during the retention interval (Ks = 1.90, 2.19, 2.05, 1.77, 1.68, 1.47, for target-present across set size 4, 8, and 12, and target-absent across the three set sizes, respectively). In summary, changes were not detected more accurately on trials in which the search target was absent compared to target-present trials.

Discussion

In Experiment 2, we found that, regardless of whether a unique target object appeared in the visual search array, approximately one half to one object’s worth of information was displaced from visual working memory based on change-detection performance being less accurate in the dual task than the memory-alone condition. If it were the case that information was displaced due to the necessity to encode the target into the visual working memory store, then performance would have been unimpaired on target-absent trials. In contrast, we found that change-detection accuracy was essentially the same on target-present and -absent trials.

The present findings support the nonspecific masking account of reduced change-detection performance when a search array is presented in two ways. First, we found that information was displaced from visual working memory even when the visual search array did not contain a unique item. This suggests that the findings from Experiment 3 of Woodman et al. (2001) were not simply due to the presence of the unique square with a gap up or down in the context of distractors with gaps to the left and right. Second, Experiment 2 casts serious doubt on the hypothesis that information is displaced from visual working memory by the encoding of targets because a similar drop in change-detection performance was found on target-absent and target-present trials.

An alternative explanation for the findings of Experiment 2 that does not appeal to the idea of nonspecific disruption or masking by the search array is that even on target-absent trials a distractor was encoded into visual working memory to program the appropriate manual response. In other words, it is possible that the computational necessity of making an absent response required that some kind of information (i.e.g., a distractor) be encoded into visual working memory. Such an account would predict that, every time a visual search array requires a response, either a target is encoded to generate the correct target-present response or a distractor is encoded (e.g., the last distractor to receive the benefit of perceptual attention) to program the correct target-absent response. In contrast, a recent study suggests that stimulus-responses mappings may be stored in long-term memory during search tasks in which the target identity does not change during the experiment (Woodman, Luck, & Schall, 2007), consistent with theories of automaticity (Logan, 1988; Logan, Taylor, & Etherton, 1999). To distinguish between these hypotheses we conducted another experiment in which participants were required to perform a go/no-go visual search task with target-present (go) and target-absent (no-go) trials.

Experiment 3

The goal of Experiment 3 was to test the hypothesis that the displacement of information from visual working memory is due to the necessity to encode at least one item from the search array so that the correct response can be generated. It should be noted that such a hypothesis would need to account for the findings of Experiment 3 in Woodman et al. (2001) by some other process, such as, the obligatory encoding of the unique search element into visual working memory even when the search arrays were passively viewed. However, the explanation that displacement occurs due to the necessity to generate a response cannot be ruled out with existing data. Consistent with the hypothesis that visual working memory processes can be interfered with by response selection, Jolicoeur and Dell’ Acqua (1998) showed that response selection in a dual-task paradigm can interfere with working memory encoding. To test the hypothesis that programming any response to a search array displaces information from visual working memory we used a design that was similar to that of Experiment 2. The difference was that the target-absent trials required no response. In the visual search task, participants responded as fast as possible when they found the search target but made no response when a target was not present in the array. We predicted that if response selection was the cause of the informational displacement from visual working memory then change-detection performance should not be impaired on the target-absent trials not requiring a response but would be worse on target-present trials. In contrast, if information is displaced from visual working memory by the presentation of any stimulus array, as the nonspecific masking account proposes, then change detection should be similarly impaired on both target-present and target-absent trials in the dual-task condition.

Method

The methods used in Experiment 3 were identical to those of Experiment 2 with the following two exceptions. First, a different group of observers participated after informed consent was obtained. Second, the participants were instructed to respond by pressing a button with the index finger on their dominant hand if a target was present and make no response on the other half of trials in which no target was present.

Results

The mean visual search accuracy on target-present trials was high (above 97.8 % correct) and did not significantly differ across conditions and set sizes (F < 1.0). The mean visual search RTs from the target-present trials in each condition are summarized in Figure 4A. Replicating a now familiar pattern, search RT was significantly slower when performed in the dual-task condition (mean = 1447.6 ms) than when performed alone (mean = 1259.2 ms) with an essentially constant delay across set sizes. The slopes of the search functions were nearly identical (72.0 ms/item and 79.4 ms/item in the dual task and search-alone conditions, respectively). The ANOVA yielded a marginally significant effect of condition (F(1,9) = 4.38, p = .066), a significant effect of set size (F(1,9) = 114.66, p < .0001), but no interaction between these factors (F < 1.0).

Figure 4.

Figure 4

Open in a new tab

The results from Experiment 3. A: The visual search RTs from the dual-task (dashed lines, empty symbols) and the search-alone condition (solid lines, filled symbols) for target present trials. Target-absent trials required no response. B: The change-detection accuracy in the memory-alone condition and the dual-task condition as a function of set size and target presence.

Mean change-detection accuracy is depicted in Figure 4B. Observers were most accurate at detecting color changes in the memory-alone condition (mean = 86.8 % correct) and less accurate during all of the dual-task trials types (means = 77.7, 77.5, 83.9, 74.4, 67.5, 73.0, on set size 4 target-present, set size 4 target-absent, set size 8 target-present, set size 8 target-absent, set size 12 target-present, and set size 12 target-absent trials, respectively). The ANOVA yielded a significant main effect of set size (F(2,18) = 4.39, p < .05) but neither the main effect of target presence nor the interaction were significant (ps > .15). Planned comparisons between change-detection performance in the memory-alone condition and during no-go trials of each set size in the dual-task condition resulted in a significant difference in memory-task accuracy between memory-alone performance and each target-absent set size (ps < .05). The K values estimating the number of objects’ worth of information stored in visual working memory during the different types of trials mirrored the change-detection performance in terms of percent correct. More objects were stored in the memory-alone condition (K = 2.94) than during any of the dual-task trials regardless of set size or target presence (Ks = 2.24, 2.21, 2.05, 1.95, 1.68, 2.01, on set size 4 target-present, set size 4 target-absent, set size 8 target-present, set size 8 target-absent, set size 12 target-present, and set size 12 target-absent trials, respectively).

Discussion

In Experiment 3, we found that change-detection accuracy was no more accurate on target-absent no-go trials in the dual-task condition than on target-present-go trials, whereas memory-task performance was better in the memory-alone condition than when either a go or no-go search array was viewed. These findings suggest that information is not displaced from visual working memory when visual search is performed due to the necessity to encode either a target or distractor object for a correct manual response to be programmed. In the present experiment, no response to a target-absent array was necessary, but we again found that information was displaced from visual working memory on these target-absent no-go trials. These results are what would be expected if nonspecific masking were displacing information from visual working memory because even the target-absent arrays without a unique search element and no response requirement caused displacement. In summary, the findings are inconsistent with what would be expected if information were being displaced from visual working memory by a search element needed to program the appropriate response on each trial.

Our use of the no-go trials to test the hypothesis that visual working memory displacement requires response selection was based on the idea that such no-go trials do not recruit response-selection mechanisms. However, the psychological refractory period (PRP) literature suggests that the presentation of stimuli requiring subjects not to respond may engage mechanisms similar to those that require a response. That is, a no-go first target stimulus can cause the slowing of the response to a second target stimulus. However, this slowing is less than that observed for first targets requiring a ‘go’ response (Pashler, 1994). This suggests that no-go stimuli engage the response selection mechanisms that are believed to cause the PRP bottleneck to a lesser extent than stimuli requiring a response. Thus, we should have expected to see less of a reduction upon visual working memory task performance on these trials, but this was not observed.

We note that there were numerical differences between change-detection accuracy on target-present versus -absent trials across visual search set sizes. However, these effects were not systematic and, thus, appear to be due to noise from the sample. With regard to the hypothesis we tested, if search targets were displacing information on target-present trials but not target-absent-no-go trials then change-detection accuracy should be higher on target-absent trials. However, we only found this pattern when the set size of the intervening search array contained 12 elements. At set size 8 we found a larger effect in the opposite direction and accuracy in the two search conditions at set size 4 were the same. In summary, the change-detection results do not support the hypothesis that information is only displaced from visual working memory when the visual search array requires that a response be generated.

A recently published study by Oh and Kim (2004) focused on the difference between maintaining spatial locations versus objects in visual working memory during visual search. However, they also used a visual search task in which targets were present or absent. They found that change-detection performance tended to be lower on target-absent compared to target-present trials. This result is the opposite of what one would expect it if displacement were due to the target representation were being encoded into visual working memory. In Experiment 2 and 3, we found similar trends at some set sizes but did not find consistent evidence for better change detection on target-present than target-absent trials. There were a number of differences between the search stimuli used in this study and that of Oh and Kim (2004), in addition to possible strategic differences between groups of participants in determining target absence (Chun & Wolfe, 1996). However, the findings of the present study are reliable as they are essentially replicated between Experiment 2 and 3. The cause of the discrepancy between the present findings and those of Oh and Kim (2004) is a topic of further investigation. Regardless, neither the present study nor that of Oh and Kim (2004) found a pattern of change-detection performance that would support the hypothesis that the encoding of a search element into visual working memory is what displaces information from visual working memory during search.

General Discussion

This study addressed the role of visual working memory in the performance of visual search tasks. Although several theories of attention imply that visual working memory would be used extensively during visual search, previous research indicated that visual search was not slowed when visual working memory was occupied by a concurrent change detection task (Woodman et al., 2001). However, memory accuracy was reduced in this dual-task condition compared to a condition in which no visual search array was presented, raising the possibility that visual working memory is indeed used during visual search. Woodman et al. (2001) proposed that this reduction in memory performance was a result of nonspecific disruption of the memory representation due to the mere appearance of the search array, but alternative explanations are possible. The present study tested three of these alternative explanations so that it would be possible to know with greater confidence whether visual working memory plays a key role in visual search.

In Experiment 1, we tested the hypothesis that information is expelled from visual working memory in anticipation of performing visual search. This was examined by eliminating the search array on a subset of trials so that anticipatory expulsion could be seen in the absence of any nonspecific disruption produced by the mere appearance of the search array. We found that, relative to the memory-alone baseline condition, change-detection performance was reduced when the search array was actually presented but it was not reduced when the search array was not presented. Because subjects could not know in advance whether or not a search array would be presented, the lack of a memory disruption when the search array was absent indicates that subjects did not preemptively displace information from visual working memory in anticipation of performing the visual search task. Instead, the appearance of the search array triggered the displacement of information. When combined with previous research showing that the displacement of information is also produced when subjects are instructed to ignore the search arrays (Woodman et al., 2001, Experiment 3), these results support the proposal that the mere appearance of the search array produces a nonspecific disruption of working memory.

In Experiment 2 and 3, we tested the hypothesis that encoding the target (or even a distractor) into visual working memory displaces information stored in visual working memory. We found no evidence to support this hypothesis. Participants’ change-detection performance was just as accurate on target-present trials as on target-absent trials. This was the case even when the target-absent array did not require a response. Thus, the present experiments rule out a number of theoretical accounts of how visual working memory’s limited representational space is utilized during visual search.

The nonspecific disruption or masking explanation accounts for the findings presented here and in Woodman et al. (2001). These findings as a whole are consistent with the idea that some of the information from the visual search arrays may force its way into visual working memory displacing the task-relevant information. The idea that we may be unable to completely prevent information from disrupting visual working memory maintenance has been demonstrated using visual working memory tasks in which the stimuli are presented in a serial fashion. For example, observers appear to be unable to keep an obviously task-irrelevant item from disrupting memory for the previously presented items in a sequence (Parmentier, Tremblay, & Jones, 2004). This visual-suffix effect bears a striking similarity to the irrelevant-picture effect in which the presentation of visual stimuli during a retention interval interferes with memory for the task-relevant information previously shown (Logie & Marchetti, 1991; Quinn & McConnell, 1996). It is interesting to note that the loss of information from visual working memory is not modulated by the set size of the search array. That is, the same amount of information appears to be displaced from visual working memory regardless of the amount of information in the visual search array. We note that in a recent dual-task study, subjects were always shown a visual search array during the retention interval of a working memory task and instructed to either search for a target in the array or ignore it (Johnson, Hollingworth, & Luck, 2008). Given these instructions, subjects showed a larger decrease in memory-task performance when actively searching and trying to maintain objects in visual working memory defined by a conjunction of features than when ignoring the search arrays. This result appears to argue against the nonspecific masking account because a search array was always presented. However, this result may also be due to nonspecific masking given that subjects likely made eye movements during search, and the visual transients that occurred at each shift of gaze may have produced a larger disruption of the contents of visual working memory. Reconciling the present findings and those of Johnson et al. (2008) are the topic of continued study.

It is useful to contrast the present pattern of results with the pattern obtained under conditions that clearly involve an interaction between visual working memory and visual attention in visual search. For example, when the search target is the same from trial to trial, as in most laboratory search experiments, it seems possible that long-term memory storage could be used to control the search process. However, working memory would presumably be necessary in many real-world situations, in which an individual might search for one object at one moment and a different object at the next moment (e.g., a jar of peanut butter and then a jar of jelly when making a peanut-butter-and-jelly sandwich). To test this possibility, Woodman et al. (2007) compared the standard laboratory situation, in which the target remains constant for an entire block of trials, with a condition in which the target varied from trial to trial and was indicated with a cue at the beginning of each trial. Memory performance declined in the dual-task condition compared to the memory-alone condition by a substantially greater amount when the target was cued on a trial-by-trial basis rather than remaining constant across trials. In addition, the visual search slope was steeper in the dual-task condition when the target identity was cued on each trial rather than remaining constant. Thus, when the visual search task clearly demands the use of visual working memory, the visual search and visual working memory tasks interfere with each other in a way that was not observed here. Similar results have been obtained when visual search is combined with a spatial working memory task (Oh & Kim, 2004; Woodman & Luck, 2004), presumably because spatial working memory is used to keep track of recently searched locations (e.g., Klein, 1988; McCarley, Wang, Kramer, Irwin, & Paterson, 2003). Similar results have also been obtained when the object working memory task used here was combined with mental rotation rather than visual search (Hyun & Luck, 2007), presumably because the representation being rotated is maintained in the object working memory system during the rotation process. Thus, the present pattern of results, in which working memory performance is slightly disrupted by the appearance of the search array, does not appear to support the proposal that visual search necessarily requires the use of object working memory.

Acknowledgements

This research was supported by individual NRSA fellowships from NIMH and NEI (F31 MH12995-01 and F32 EY015043) to G.F.W.

Footnotes

1

Several models of attention and a number of studies of temporary memory use the term visual short-term memory instead of visual working memory. Because these terms appear to refer to the same construct, the term visual working memory is used to refer to the temporary memory store for visual object representations throughout this paper.

References

  1. Breitmeyer B. Visual masking: An integrative approach. Oxford: Oxford University Press; 1984. [Google Scholar]
  2. Bundesen C. A theory of visual attention. Psychological Review. 1990;97:523–547. doi: 10.1037/0033-295x.97.4.523. [DOI] [PubMed] [Google Scholar]
  3. Chelazzi L, Duncan J, Miller EK, Desimone R. Responses of neurons in inferior temporal cortex during memory-guided visual search. Journal of Neurophysiology. 1998;80:2918–2940. doi: 10.1152/jn.1998.80.6.2918. [DOI] [PubMed] [Google Scholar]
  4. Chelazzi L, Miller EK, Duncan J, Desimone R. A neural basis for visual search in inferior temporal cortex. Nature. 1993;363:345–347. doi: 10.1038/363345a0. [DOI] [PubMed] [Google Scholar]
  5. Chun MM, Wolfe JM. Just say no: How are visual searches terminated when there is no target present? Cognitive Psychology. 1996;30(1):39–78. doi: 10.1006/cogp.1996.0002. [DOI] [PubMed] [Google Scholar]
  6. Cohen A, Ivry RB. Density effects in conjunction search: Evidence for a coarse location mechanism of feature integration. Journal of Experimental Psychology: Human Perception and Performance. 1991;17:891–901. doi: 10.1037//0096-1523.17.4.891. [DOI] [PubMed] [Google Scholar]
  7. Cowan N. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences. 2001;24:87–185. doi: 10.1017/s0140525x01003922. [DOI] [PubMed] [Google Scholar]
  8. Desimone R, Duncan J. Neural mechanisms of selective visual attention. Annual Review of Neuroscience. 1995;18:193–222. doi: 10.1146/annurev.ne.18.030195.001205. [DOI] [PubMed] [Google Scholar]
  9. Duncan J, Humphreys GW. Visual search and stimulus similarity. Psychological Review. 1989;96(3):433–458. doi: 10.1037/0033-295x.96.3.433. [DOI] [PubMed] [Google Scholar]
  10. Hyun J-S, Luck SJ. Visual working memory as the substrate for mental rotation. Psychonomic Bulletin & Review. 2007;14:154–158. doi: 10.3758/bf03194043. [DOI] [PubMed] [Google Scholar]
  11. Johnson JS, Hollingworth A, Luck SJ. The role of attention in the maintenance of feature bindings in visual short-term memory. Journal of Experimental Psychology: Human Perception and Performance. 2008;34:41–55. doi: 10.1037/0096-1523.34.1.41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Jolicoeur P, Dell' Acqua R. The demonstration of short-term consolidation. Cognitive Psychology. 1998;36(2):138–202. doi: 10.1006/cogp.1998.0684. [DOI] [PubMed] [Google Scholar]
  13. Klein R. Inhibitory tagging system facilitates visual search. Nature. 1988;334:430–431. doi: 10.1038/334430a0. [DOI] [PubMed] [Google Scholar]
  14. Loftus GR, Loftus EF. Essence of Statistics. 2nd ed. New York: Random House; 1988. [Google Scholar]
  15. Logan GD. Attention in character classification tasks: Evidence for the automaticity of component stages. Journal of Experimental Psychology: General. 1978;107:32–63. [Google Scholar]
  16. Logan GD. Toward an instance theory of automatization. Psychological Review. 1988;95:492–527. [Google Scholar]
  17. Logan GD, Taylor SE, Etherton JL. Attention and automaticity: Toward a theoretical integration. Psychological Research. 1999;62:165–181. [Google Scholar]
  18. Logie RH, Marchetti C. Visuo-spatial working memory: Visual, spatial or central executive. In: Logie RH, Denis M, editors. Mental images in human cognition. Amsterdam: North Holland Press; 1991. pp. 105–115. [Google Scholar]
  19. McCarley JS, Wang RF, Kramer AF, Irwin DA, Paterson MS. How much memory does oculomotor search have? Psychological Science. 2003;14:422–426. doi: 10.1111/1467-9280.01457. [DOI] [PubMed] [Google Scholar]
  20. Oh SH, Kim MS. The role of spatial working memory in visual search efficiency. Psychonomic Bulletin and Review. 2004;11(2):275–281. doi: 10.3758/bf03196570. [DOI] [PubMed] [Google Scholar]
  21. Olivers CNL, Meijer F, Theeuwes J. Feature-based memory-driven attentional capture: Visual working memory contents affects visual attention. Journal of Experimental Psychology: Human Perception and Performance. 2006;32:1243–1265. doi: 10.1037/0096-1523.32.5.1243. [DOI] [PubMed] [Google Scholar]
  22. Parmentier FBF, Tremblay S, Jones DM. Exploring the suffix effect in serial visuospatial short-term memory. Psychonomic Bulletin and Review. 2004;11(2):289–295. doi: 10.3758/bf03196572. [DOI] [PubMed] [Google Scholar]
  23. Pashler H. Familiarity and visual change detection. Perception & Psychophysics. 1988;44:369–378. doi: 10.3758/bf03210419. [DOI] [PubMed] [Google Scholar]
  24. Pashler H. Dual-task interference in simple tasks: Data and theory. Psychological Bulletin. 1994;116:220–244. doi: 10.1037/0033-2909.116.2.220. [DOI] [PubMed] [Google Scholar]
  25. Quinn JG, McConnell J. Irrelevant pictures in visual working memory. Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology. 1996;49A(1):200–215. doi: 10.1080/713755613. [DOI] [PubMed] [Google Scholar]
  26. Soto D, Heinke D, Humphreys GW, Blanco MJ. Early, involuntary top-down guidance of attention from working memory. Journal of Experimental Psychology: Human Perception and Performance. 2005;31(2):248–261. doi: 10.1037/0096-1523.31.2.248. [DOI] [PubMed] [Google Scholar]
  27. Vogel EK, Woodman GF, Luck SJ. Storage of features, conjunctions, and objects in visual working memory. Journal of Experimental Psychology: Human Perception and Performance. 2001;27:92–114. doi: 10.1037//0096-1523.27.1.92. [DOI] [PubMed] [Google Scholar]
  28. Woodman GF. Unpublished Dissertation. Iowa City, IA: The University of Iowa; 2002. The involvement of visual working memory in visual search. [Google Scholar]
  29. Woodman GF, Luck SJ. Visual search is slowed when visuospatial working memory is occupied. Psychonomic Bulletin and Review. 2004;11(2):269–274. doi: 10.3758/bf03196569. [DOI] [PubMed] [Google Scholar]
  30. Woodman GF, Luck SJ. Do the contents of visual working memory automatically influence attentional selection during visual search? Journal of Experimental Psychology: Human Perception and Performance. 2007;33:363–377. doi: 10.1037/0096-1523.33.2.363. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Woodman GF, Luck SJ, Schall JD. The role of working memory representations in the control of attention. Cerebral Cortex. 2007;17:i118–i124. doi: 10.1093/cercor/bhm065. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Woodman GF, Vogel EK, Luck SJ. Visual search remains efficient when visual working memory is full. Psychological Science. 2001;12:219–224. doi: 10.1111/1467-9280.00339. [DOI] [PubMed] [Google Scholar]

RESOURCES