The idea that most of us are good at recognizing faces permeates everyday thinking and is widely used in the research literature. However, it is a correct characterization only of familiar-face recognition. In contrast, the perception and recognition of unfamiliar faces can be surprisingly error-prone, and this has important consequences in many real-life settings. We emphasize the variability in views of faces encountered in everyday life and point out how neglect of this important property has generated some misleading conclusions. Many approaches have treated image variability as unwanted noise, whereas we show how studies that use and explore the implications of image variability can drive substantial theoretical advances.