Multimodal Representation and Intermodal Similarity

Cues of Space in the Audio Description of Film

Research output: ThesisDoctoral ThesisCollection of Articles

Abstract

The present dissertation analyses the representation of space in filmic audio description. The main objective of this study is to shed light on two critical challenges for audio description: (1) how the filmic space becomes audible in audio-described film through spoken language, sound effects, and music, and (2) how the visual representation of space in film can be cued by the linguistic mode in an audio description.

This dissertation consists of four articles and a thesis summary. The first article focusses on the multimodal representation of space through auditory cues. The other three articles explore intermodal similarity, which involves the question of how language reflects visual representation. To explain the complex phenomenon of translating images into words, this study applies the theoretical and analytical tools from translation studies, film studies and cognitive linguistics, and also adopts a cognitive orientation to explain both the filmic and linguistic representations as being cognitive representations that are constructed through visual, auditory, and linguistic cues.

This research first establishes that the auditory multimodality of an audio-described film creates a variant of the multimodality of the audiovisual film that entails dynamic constellations, perspectives and foci. Secondly, the analysis presents evidence for the varied potential of the linguistic mode in terms of representing space. Thirdly, this study defines strategies for intermodal similarity in terms of the corresponding linguistic signs for the filmic cues of representation and narration. The results of this dissertation may be valuable in justifying the consistency and standards for audio description as well as in developing other, more far-reaching usages of audio description as a mode of transforming information from one system of representation into another.
Original languageEnglish
Place of PublicationHelsinki
Publisher
Print ISBNs978-951-51-0368-0
Electronic ISBNs978-951-51-0369-7
Publication statusPublished - 2014
MoE publication typeG5 Doctoral dissertation (article)

Fields of Science

  • 6160 Other humanities
  • Translation Studies
  • Blindness
  • 6121 Languages
  • Linguistics
  • 6131 Theatre, dance, music, other performing arts
  • Film studies

Cite this

@phdthesis{c815e93b8bd94eca99bf385c25673a24,
title = "Multimodal Representation and Intermodal Similarity: Cues of Space in the Audio Description of Film",
abstract = "The present dissertation analyses the representation of space in filmic audio description. The main objective of this study is to shed light on two critical challenges for audio description: (1) how the filmic space becomes audible in audio-described film through spoken language, sound effects, and music, and (2) how the visual representation of space in film can be cued by the linguistic mode in an audio description. This dissertation consists of four articles and a thesis summary. The first article focusses on the multimodal representation of space through auditory cues. The other three articles explore intermodal similarity, which involves the question of how language reflects visual representation. To explain the complex phenomenon of translating images into words, this study applies the theoretical and analytical tools from translation studies, film studies and cognitive linguistics, and also adopts a cognitive orientation to explain both the filmic and linguistic representations as being cognitive representations that are constructed through visual, auditory, and linguistic cues. This research first establishes that the auditory multimodality of an audio-described film creates a variant of the multimodality of the audiovisual film that entails dynamic constellations, perspectives and foci. Secondly, the analysis presents evidence for the varied potential of the linguistic mode in terms of representing space. Thirdly, this study defines strategies for intermodal similarity in terms of the corresponding linguistic signs for the filmic cues of representation and narration. The results of this dissertation may be valuable in justifying the consistency and standards for audio description as well as in developing other, more far-reaching usages of audio description as a mode of transforming information from one system of representation into another.",
keywords = "6160 Other humanities, Translation Studies, Blindness, 6121 Languages, Linguistics, 6131 Theatre, dance, music, other performing arts, Film studies",
author = "Maija Hirvonen",
note = "Nelj{\"a} artikkelia ja tiivistelm{\"a}osa. Volume: Proceeding volume:",
year = "2014",
language = "English",
isbn = "978-951-51-0368-0",
publisher = "University of Helsinki",
address = "Finland",

}

Multimodal Representation and Intermodal Similarity : Cues of Space in the Audio Description of Film. / Hirvonen, Maija.

Helsinki : University of Helsinki, 2014. 133 p.

Research output: ThesisDoctoral ThesisCollection of Articles

TY - THES

T1 - Multimodal Representation and Intermodal Similarity

T2 - Cues of Space in the Audio Description of Film

AU - Hirvonen, Maija

N1 - Neljä artikkelia ja tiivistelmäosa. Volume: Proceeding volume:

PY - 2014

Y1 - 2014

N2 - The present dissertation analyses the representation of space in filmic audio description. The main objective of this study is to shed light on two critical challenges for audio description: (1) how the filmic space becomes audible in audio-described film through spoken language, sound effects, and music, and (2) how the visual representation of space in film can be cued by the linguistic mode in an audio description. This dissertation consists of four articles and a thesis summary. The first article focusses on the multimodal representation of space through auditory cues. The other three articles explore intermodal similarity, which involves the question of how language reflects visual representation. To explain the complex phenomenon of translating images into words, this study applies the theoretical and analytical tools from translation studies, film studies and cognitive linguistics, and also adopts a cognitive orientation to explain both the filmic and linguistic representations as being cognitive representations that are constructed through visual, auditory, and linguistic cues. This research first establishes that the auditory multimodality of an audio-described film creates a variant of the multimodality of the audiovisual film that entails dynamic constellations, perspectives and foci. Secondly, the analysis presents evidence for the varied potential of the linguistic mode in terms of representing space. Thirdly, this study defines strategies for intermodal similarity in terms of the corresponding linguistic signs for the filmic cues of representation and narration. The results of this dissertation may be valuable in justifying the consistency and standards for audio description as well as in developing other, more far-reaching usages of audio description as a mode of transforming information from one system of representation into another.

AB - The present dissertation analyses the representation of space in filmic audio description. The main objective of this study is to shed light on two critical challenges for audio description: (1) how the filmic space becomes audible in audio-described film through spoken language, sound effects, and music, and (2) how the visual representation of space in film can be cued by the linguistic mode in an audio description. This dissertation consists of four articles and a thesis summary. The first article focusses on the multimodal representation of space through auditory cues. The other three articles explore intermodal similarity, which involves the question of how language reflects visual representation. To explain the complex phenomenon of translating images into words, this study applies the theoretical and analytical tools from translation studies, film studies and cognitive linguistics, and also adopts a cognitive orientation to explain both the filmic and linguistic representations as being cognitive representations that are constructed through visual, auditory, and linguistic cues. This research first establishes that the auditory multimodality of an audio-described film creates a variant of the multimodality of the audiovisual film that entails dynamic constellations, perspectives and foci. Secondly, the analysis presents evidence for the varied potential of the linguistic mode in terms of representing space. Thirdly, this study defines strategies for intermodal similarity in terms of the corresponding linguistic signs for the filmic cues of representation and narration. The results of this dissertation may be valuable in justifying the consistency and standards for audio description as well as in developing other, more far-reaching usages of audio description as a mode of transforming information from one system of representation into another.

KW - 6160 Other humanities

KW - Translation Studies

KW - Blindness

KW - 6121 Languages

KW - Linguistics

KW - 6131 Theatre, dance, music, other performing arts

KW - Film studies

M3 - Doctoral Thesis

SN - 978-951-51-0368-0

PB - University of Helsinki

CY - Helsinki

ER -