The effects of temporal synchrony and visual material on audio-visual integration

Maria Mittag, Rika Takegata, Teija Kujala

Forskningsoutput: Kapitel i bok/rapport/konferenshandlingKonferensbidragProfessionell


Associating letters with speech sounds is essential when learning to read. Recent studies on audio-visual integration suggested that 1) the integration of letter and speech sound takes place automatically and 2) that temporal synchrony seems to be critical for the integration. In the current study, we wished to determine the neural networks associated with an automatic integration of written and heard syllables in literate adults. The mismatch negativity (MMN) was recorded in a new multi-feature paradigm whereby the integration of several speech sounds with letters can be addressed within one experimental session. Subjects were presented with auditory stimuli (consonant-vowel syllables) together with visual stimuli, which differed between conditions. The auditory stimuli included vowel or consonant changes, or changes in syllable intensity, frequency, or vowel length. In the experimental condition the visual stimuli were written syllables, whereas in the control condition, they were scrambled images of the written syllables. The visual stimuli were presented in half of the blocks synchronously with the auditory stimuli and in the other half with a time delay. During the stimulation, the subjects pressed a button to designated visual targets. Changes in auditory stimuli elicited a MMN/N2 response, in all conditions. The topography of this response was different in the stimulus blocks including syllables than in those including scrambled images. Furthermore, topographic differences were observed between the conditions with and without the time delay. The results indicated that different neural networks are activated when speech stimuli are presented together with linguistic than with non-linguistic visual stimuli and that this effect seems to be influenced by temporal synchrony.
Titel på gästpublikationFrontiers Human Neuroscience Conference Abstract : MMN 09 Fifth Conference on Mismatch Negativity (MMN) and its Clinical and Scientific Applications
Antal sidor1
FörlagFrontiers Media S.A.
Utgivningsdatum26 mar 2009
StatusPublicerad - 26 mar 2009
MoE-publikationstypD3 Professionella konferenshandlingar
EvenemangConference on Mismatch Negativity (MMN) and its Clinical and Scientific Applications (MMN 09) - Budapest, Ungern
Varaktighet: 4 apr 20097 apr 2009
Konferensnummer: 5th


  • 515 Psykologi

Citera det här