Visual influences on the perception of emotion in music

Musicians use facial expressions, expressive gestures, and body movements in their performances. The physical demands of sound production account for much of this aspect of performance, but certain movements and facial expressions are introduced for expressive purposes. We investigated whether facial expressions that accompany music performances influence the perception of emotion in music. Participants watched a video recording of a performer singing a melodic interval. They then judged the emotion conveyed by the music. The interval was either a major third or a minor third. The auditory component was paired with the original facial expression, or it was synchronized with the facial expression of another sung interval. This manipulation resulted in four conditions: major audio with major visual; major audio with minor visual; minor audio with major visual; and minor audio with minor visual. Judgments of emotional meaning were strongly influenced by both audio and visual information, even though participants were instructed to attend only to the music. Overall, judgments of emotional meaning were more strongly influenced by facial expressions than by the music. We discuss the implications of the findings for theories of musical emotion.