Audio and visual art accompany one-another in many contexts – such as films, television, advertising, and gaming. Many scientific studies have researched how music affects our perception of visual information. We know, for example, that film soundtracks can add meaning to characters’ actions and affect which items from a scene are remembered best. However, few have explored the inverse relationship – whether visual information influences our perception and memory of the music.
In the early 1980’s, MTV popularised a new genre of art: the music video. Some researchers claimed that these accompanying visuals would decrease the appeal of the music by distracting or limiting the listener’s imagination. Others believed that the videos would enhance a song’s appeal by adding deeper meaning to the music. Whilst the debate continues, research does suggest that music videos can prevent “wear out” – becoming tired of a song through repeated exposure.
Researchers Marilyn Boltz, Brittany Ebendorf, and Benjamin Field set out to better understand whether moving image affects our impression of music. Their results suggest that it does.
The experimenters used two types of visuals of accompany the music: videos and image montages. Generally, the mere presence of visuals changed how the music was perceived. Melodies accompanied by visuals were perceived to be faster, louder, and more active and rhythmic than the same melodies heard by themselves.
Images also seemed to intensify the emotional impact of a tune. Melodies that had been judged individually as neutral seemed to take on positive or negative characteristics, depending on the characteristics of the accompanying visuals. The perceived acoustic qualities of a tune (its speed, loudness, and rhythms) also changed, depending on whether it was accompanied by positive or negative visuals.
Finally, visual changes distorted melody recognition depending on whether images had positive or negative characteristics. Negative displays decreased recognition, whilst positive displays increased it.
Music for media is traditionally an afterthought. For example, Millward Brown estimate that brands spend a 84% of time and money on visual branding. This present study, along with many others, highlights the importance of considering audio-visual media as interactive. Content creators must not only how the accompanying music might affect their visuals, but also how these visuals might change perceptions the accompanying music.