Neural Correlates of Temporal Complexity and Synchrony during Audiovisual Correspondence Detection

Oliver Baumann, Joyce M G Vromen, Allen Cheung, Jessica McFadyen, Yudan Ren, Christine C Guo

Research output: Contribution to journalArticleResearchpeer-review

10 Downloads (Pure)

Abstract

We often perceive real-life objects as multisensory cues through space and time. A key challenge for audiovisual integration is to match neural signals that not only originate from different sensory modalities but also that typically reach the observer at slightly different times. In humans, complex, unpredictable audiovisual streams lead to higher levels of perceptual coherence than predictable, rhythmic streams. In addition, perceptual coherence for complex signals seems less affected by increased asynchrony between visual and auditory modalities than for simple signals. Here, we used functional magnetic resonance imaging to determine the human neural correlates of audiovisual signals with different levels of temporal complexity and synchrony. Our study demonstrated that greater perceptual asynchrony and lower signal complexity impaired performance in an audiovisual coherence-matching task. Differences in asynchrony and complexity were also underpinned by a partially different set of brain regions. In particular, our results suggest that, while regions in the dorsolateral prefrontal cortex (DLPFC) were modulated by differences in memory load due to stimulus asynchrony, areas traditionally thought to be involved in speech production and recognition, such as the inferior frontal and superior temporal cortex, were modulated by the temporal complexity of the audiovisual signals. Our results, therefore, indicate specific processing roles for different subregions of the fronto-temporal cortex during audiovisual coherence detection.

Original languageEnglish
Article numbere0294-17.2018
Number of pages13
JournaleNeuro
Volume5
Issue number1
DOIs
Publication statusPublished - 23 Jan 2018
Externally publishedYes

Fingerprint

Temporal Lobe
Prefrontal Cortex
Cues
Magnetic Resonance Imaging
Brain
Recognition (Psychology)

Cite this

Baumann, Oliver ; Vromen, Joyce M G ; Cheung, Allen ; McFadyen, Jessica ; Ren, Yudan ; Guo, Christine C. / Neural Correlates of Temporal Complexity and Synchrony during Audiovisual Correspondence Detection. In: eNeuro. 2018 ; Vol. 5, No. 1.
@article{0c087123a954463287a1d812896a3d41,
title = "Neural Correlates of Temporal Complexity and Synchrony during Audiovisual Correspondence Detection",
abstract = "We often perceive real-life objects as multisensory cues through space and time. A key challenge for audiovisual integration is to match neural signals that not only originate from different sensory modalities but also that typically reach the observer at slightly different times. In humans, complex, unpredictable audiovisual streams lead to higher levels of perceptual coherence than predictable, rhythmic streams. In addition, perceptual coherence for complex signals seems less affected by increased asynchrony between visual and auditory modalities than for simple signals. Here, we used functional magnetic resonance imaging to determine the human neural correlates of audiovisual signals with different levels of temporal complexity and synchrony. Our study demonstrated that greater perceptual asynchrony and lower signal complexity impaired performance in an audiovisual coherence-matching task. Differences in asynchrony and complexity were also underpinned by a partially different set of brain regions. In particular, our results suggest that, while regions in the dorsolateral prefrontal cortex (DLPFC) were modulated by differences in memory load due to stimulus asynchrony, areas traditionally thought to be involved in speech production and recognition, such as the inferior frontal and superior temporal cortex, were modulated by the temporal complexity of the audiovisual signals. Our results, therefore, indicate specific processing roles for different subregions of the fronto-temporal cortex during audiovisual coherence detection.",
author = "Oliver Baumann and Vromen, {Joyce M G} and Allen Cheung and Jessica McFadyen and Yudan Ren and Guo, {Christine C}",
year = "2018",
month = "1",
day = "23",
doi = "10.1523/ENEURO.0294-17.2018",
language = "English",
volume = "5",
journal = "eNeuro",
issn = "2373-2822",
publisher = "Society for Neuroscience",
number = "1",

}

Neural Correlates of Temporal Complexity and Synchrony during Audiovisual Correspondence Detection. / Baumann, Oliver; Vromen, Joyce M G; Cheung, Allen; McFadyen, Jessica; Ren, Yudan; Guo, Christine C.

In: eNeuro, Vol. 5, No. 1, e0294-17.2018, 23.01.2018.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Neural Correlates of Temporal Complexity and Synchrony during Audiovisual Correspondence Detection

AU - Baumann, Oliver

AU - Vromen, Joyce M G

AU - Cheung, Allen

AU - McFadyen, Jessica

AU - Ren, Yudan

AU - Guo, Christine C

PY - 2018/1/23

Y1 - 2018/1/23

N2 - We often perceive real-life objects as multisensory cues through space and time. A key challenge for audiovisual integration is to match neural signals that not only originate from different sensory modalities but also that typically reach the observer at slightly different times. In humans, complex, unpredictable audiovisual streams lead to higher levels of perceptual coherence than predictable, rhythmic streams. In addition, perceptual coherence for complex signals seems less affected by increased asynchrony between visual and auditory modalities than for simple signals. Here, we used functional magnetic resonance imaging to determine the human neural correlates of audiovisual signals with different levels of temporal complexity and synchrony. Our study demonstrated that greater perceptual asynchrony and lower signal complexity impaired performance in an audiovisual coherence-matching task. Differences in asynchrony and complexity were also underpinned by a partially different set of brain regions. In particular, our results suggest that, while regions in the dorsolateral prefrontal cortex (DLPFC) were modulated by differences in memory load due to stimulus asynchrony, areas traditionally thought to be involved in speech production and recognition, such as the inferior frontal and superior temporal cortex, were modulated by the temporal complexity of the audiovisual signals. Our results, therefore, indicate specific processing roles for different subregions of the fronto-temporal cortex during audiovisual coherence detection.

AB - We often perceive real-life objects as multisensory cues through space and time. A key challenge for audiovisual integration is to match neural signals that not only originate from different sensory modalities but also that typically reach the observer at slightly different times. In humans, complex, unpredictable audiovisual streams lead to higher levels of perceptual coherence than predictable, rhythmic streams. In addition, perceptual coherence for complex signals seems less affected by increased asynchrony between visual and auditory modalities than for simple signals. Here, we used functional magnetic resonance imaging to determine the human neural correlates of audiovisual signals with different levels of temporal complexity and synchrony. Our study demonstrated that greater perceptual asynchrony and lower signal complexity impaired performance in an audiovisual coherence-matching task. Differences in asynchrony and complexity were also underpinned by a partially different set of brain regions. In particular, our results suggest that, while regions in the dorsolateral prefrontal cortex (DLPFC) were modulated by differences in memory load due to stimulus asynchrony, areas traditionally thought to be involved in speech production and recognition, such as the inferior frontal and superior temporal cortex, were modulated by the temporal complexity of the audiovisual signals. Our results, therefore, indicate specific processing roles for different subregions of the fronto-temporal cortex during audiovisual coherence detection.

UR - http://www.scopus.com/inward/record.url?scp=85040973650&partnerID=8YFLogxK

U2 - 10.1523/ENEURO.0294-17.2018

DO - 10.1523/ENEURO.0294-17.2018

M3 - Article

VL - 5

JO - eNeuro

JF - eNeuro

SN - 2373-2822

IS - 1

M1 - e0294-17.2018

ER -