The concept of perceptual memory refers to the neural and cognitive processes underlying the storage of specific stimulus features such as spatial frequency, orientation, shape, contrast, and color. Psychophysical studies of perceptual memory indicate that observers can retain visual information about the spatial frequency of Gabor patterns independent of the orientation with which they are presented. Compared to discrimination of gratings with the same orientation, reaction times to orthogonally oriented gratings, however, increase suggesting additional processing. Using event-related fMRI we examined the pattern of neural activation evoked when subjects discriminated the spatial frequency of Gabors presented with the same or orthogonal orientation. Blood-oxygen level dependent BOLD fMRI revealed significantly elevated bilateral activity in visual areas (V1, V2) when the gratings to be compared had an orthogonal orientation, compared to when they had the same orientation. These findings suggest that a change in an irrelevant stimulus dimension requires additional processing in primary and secondary visual areas. The finding that the task-irrelevant stimulus property (orientation) had no significant effect on the prefrontal and intraparietal cortex supports a model of working memory in which discrimination and retention of basic stimulus dimensions is based on low-level perceptual memory stores that are located at an early stage in the visual process. Our findings suggest that accessing different stores requires time and has higher metabolic costs.