According to a widely used theoretical perspective, our everyday experiences lead us to become natural experts at perceiving and recognising human faces. However, there has been considerable debate about this view. We discuss criteria for expertise and show how the debate over face expertise has often missed key points concerning the role and nature of face familiarity. For identity recognition, most of us show only limited expertise with unfamiliar faces. Carefully evaluating the senses in which it is appropriate or inappropriate to assert that we are face experts leads to the conclusion that we are, in effect, familiar face experts.