The effects of visual material and temporal synchrony on the processing of letters and speech sounds

Tutkimustuotos: ArtikkelijulkaisuArtikkeliTieteellinenvertaisarvioitu

Kuvaus

Associating letters with speech sounds is essential for reading skill acquisition. In the current study, we aimed at determining the effects of different types of visual material and temporal synchrony on the integration of letters and speech sounds. To this end, we recorded the mismatch negativity (MMN), an index of automatic change detection in the brain, from literate adults. Subjects were presented with auditory consonant-vowel syllable stimuli together with visual stimuli, which were either written syllables or scrambled pictures of the written syllables. The visual stimuli were presented in half of the blocks synchronously with the auditory stimuli and in the other half 200 ms before the auditory stimuli. The auditory stimuli were consonant, vowel or vowel length changes, or changes in syllable frequency or intensity presented by using the multi-feature paradigm. Changes in the auditory stimuli elicited MMNs in all conditions. MMN amplitudes for the consonant and frequency changes were generally larger for the sounds presented with written syllables than with scrambled syllables. Time delay diminished the MMN amplitude for all deviants. The results suggest that speech sound processing is modulated when the sounds are presented with letters versus non-linguistic visual stimuli, and further, that the integration of letters and speech sounds seems to be dependent on precise temporal alignment. Moreover, the results indicate that with our paradigm, a variety of parameters relevant and irrelevant for reading can be tested within one experiment.
Alkuperäiskielienglanti
LehtiExperimental Brain Research
Vuosikerta211
Numero2
Sivut287-298
Sivumäärä12
ISSN0014-4819
DOI - pysyväislinkit
TilaJulkaistu - 2011
OKM-julkaisutyyppiA1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä, vertaisarvioitu

Tieteenalat

  • 516 Kasvatustieteet
  • 515 Psykologia
  • 6121 Kielitieteet
  • 3112 Neurotieteet

Lainaa tätä

@article{ee059cc4d70a41b88e947eeeb1ba6afe,
title = "The effects of visual material and temporal synchrony on the processing of letters and speech sounds",
abstract = "Associating letters with speech sounds is essential for reading skill acquisition. In the current study, we aimed at determining the effects of different types of visual material and temporal synchrony on the integration of letters and speech sounds. To this end, we recorded the mismatch negativity (MMN), an index of automatic change detection in the brain, from literate adults. Subjects were presented with auditory consonant-vowel syllable stimuli together with visual stimuli, which were either written syllables or scrambled pictures of the written syllables. The visual stimuli were presented in half of the blocks synchronously with the auditory stimuli and in the other half 200 ms before the auditory stimuli. The auditory stimuli were consonant, vowel or vowel length changes, or changes in syllable frequency or intensity presented by using the multi-feature paradigm. Changes in the auditory stimuli elicited MMNs in all conditions. MMN amplitudes for the consonant and frequency changes were generally larger for the sounds presented with written syllables than with scrambled syllables. Time delay diminished the MMN amplitude for all deviants. The results suggest that speech sound processing is modulated when the sounds are presented with letters versus non-linguistic visual stimuli, and further, that the integration of letters and speech sounds seems to be dependent on precise temporal alignment. Moreover, the results indicate that with our paradigm, a variety of parameters relevant and irrelevant for reading can be tested within one experiment.",
keywords = "Audiovisual processing, Mismatch negativity (MMN), Event-related potentials (ERPs), Letter-speech sound processing, Multisensory integration, MISMATCH NEGATIVITY MMN, AUDITORY SENSORY MEMORY, SELECTIVE-ATTENTION, BRAIN RESPONSES, HEMISPHERIC LATERALIZATION, PHONEME REPRESENTATIONS, LANGUAGE EXPERIENCE, NATIVE SPEAKERS, DISCRIMINATION, INFORMATION, 516 Educational sciences, 515 Psychology, 6121 Languages, 3112 Neurosciences",
author = "Maria Mittag and Rika Takegata and Teija Kujala",
year = "2011",
doi = "10.1007/s00221-011-2686-z",
language = "English",
volume = "211",
pages = "287--298",
journal = "Experimental Brain Research",
issn = "0014-4819",
publisher = "Springer",
number = "2",

}

The effects of visual material and temporal synchrony on the processing of letters and speech sounds. / Mittag, Maria; Takegata, Rika; Kujala, Teija.

julkaisussa: Experimental Brain Research, Vuosikerta 211, Nro 2, 2011, s. 287-298.

Tutkimustuotos: ArtikkelijulkaisuArtikkeliTieteellinenvertaisarvioitu

TY - JOUR

T1 - The effects of visual material and temporal synchrony on the processing of letters and speech sounds

AU - Mittag, Maria

AU - Takegata, Rika

AU - Kujala, Teija

PY - 2011

Y1 - 2011

N2 - Associating letters with speech sounds is essential for reading skill acquisition. In the current study, we aimed at determining the effects of different types of visual material and temporal synchrony on the integration of letters and speech sounds. To this end, we recorded the mismatch negativity (MMN), an index of automatic change detection in the brain, from literate adults. Subjects were presented with auditory consonant-vowel syllable stimuli together with visual stimuli, which were either written syllables or scrambled pictures of the written syllables. The visual stimuli were presented in half of the blocks synchronously with the auditory stimuli and in the other half 200 ms before the auditory stimuli. The auditory stimuli were consonant, vowel or vowel length changes, or changes in syllable frequency or intensity presented by using the multi-feature paradigm. Changes in the auditory stimuli elicited MMNs in all conditions. MMN amplitudes for the consonant and frequency changes were generally larger for the sounds presented with written syllables than with scrambled syllables. Time delay diminished the MMN amplitude for all deviants. The results suggest that speech sound processing is modulated when the sounds are presented with letters versus non-linguistic visual stimuli, and further, that the integration of letters and speech sounds seems to be dependent on precise temporal alignment. Moreover, the results indicate that with our paradigm, a variety of parameters relevant and irrelevant for reading can be tested within one experiment.

AB - Associating letters with speech sounds is essential for reading skill acquisition. In the current study, we aimed at determining the effects of different types of visual material and temporal synchrony on the integration of letters and speech sounds. To this end, we recorded the mismatch negativity (MMN), an index of automatic change detection in the brain, from literate adults. Subjects were presented with auditory consonant-vowel syllable stimuli together with visual stimuli, which were either written syllables or scrambled pictures of the written syllables. The visual stimuli were presented in half of the blocks synchronously with the auditory stimuli and in the other half 200 ms before the auditory stimuli. The auditory stimuli were consonant, vowel or vowel length changes, or changes in syllable frequency or intensity presented by using the multi-feature paradigm. Changes in the auditory stimuli elicited MMNs in all conditions. MMN amplitudes for the consonant and frequency changes were generally larger for the sounds presented with written syllables than with scrambled syllables. Time delay diminished the MMN amplitude for all deviants. The results suggest that speech sound processing is modulated when the sounds are presented with letters versus non-linguistic visual stimuli, and further, that the integration of letters and speech sounds seems to be dependent on precise temporal alignment. Moreover, the results indicate that with our paradigm, a variety of parameters relevant and irrelevant for reading can be tested within one experiment.

KW - Audiovisual processing

KW - Mismatch negativity (MMN)

KW - Event-related potentials (ERPs)

KW - Letter-speech sound processing

KW - Multisensory integration

KW - MISMATCH NEGATIVITY MMN

KW - AUDITORY SENSORY MEMORY

KW - SELECTIVE-ATTENTION

KW - BRAIN RESPONSES

KW - HEMISPHERIC LATERALIZATION

KW - PHONEME REPRESENTATIONS

KW - LANGUAGE EXPERIENCE

KW - NATIVE SPEAKERS

KW - DISCRIMINATION

KW - INFORMATION

KW - 516 Educational sciences

KW - 515 Psychology

KW - 6121 Languages

KW - 3112 Neurosciences

U2 - 10.1007/s00221-011-2686-z

DO - 10.1007/s00221-011-2686-z

M3 - Article

VL - 211

SP - 287

EP - 298

JO - Experimental Brain Research

JF - Experimental Brain Research

SN - 0014-4819

IS - 2

ER -