We present three experiments in which subjects were asked to make speeded sex judgements (Experiment 1) or semantic judgements (Experiments 2 and 3) to face targets and nonface items, while ignoring a solitary flanking distractor face or a nonface stimulus. Distractors could be either congruent (same response category) or incongruent (different response category) with the target. Distractor congruency effects were consistently observed in all combinations of target-distractor stimulus pairs, except when a distractor face flanked a target face. The failure to find congruency effects in this condition was explored further in a fourth experiment, in which four task-irrelevant flankers were simultaneously presented. Once again, no face-face congruency effects were found, even though comparison distractors interfered with face and nonface targets alike. However, four simultaneously presented distractor faces did not interfere with nonface targets either. We suggest that these experiments demonstrate a capacity limit for visual processing in these conditions, such that no more than one face is processed at a time.