Print interpreting: process, comprehensibility, and technology

The aim of this project is to study the real-time transmission between two communication modes, speech and writing in human interaction with a method called "print interpreting". It means translation of spoken language and accompanying significant audible information into written text simultaneously with the talk. The text is typed on a computer and displayed on a screen where the letter-by-letter emerging text is visible. Print interpreting is needed as a communication aid for people with hearing disability to give them access to the speech. Since they have acquired the language in a hearing speech culture and usually can speak it, they need an interpretation which is as close as possible to the original speech. This interpretation must also give an impression of the speaker and the linguistic variation. The challenges of print interpreting are the demands of simultaneity (requiring a high production rate) and verbatim transcription. Another important challenge is to transfer all the relevant auditory information (including non-language sounds from surroundings, etc.) into a visible modality which is understandable to the hearing impaired.

The objectives of the study are 1) to investigate the process of print interpreting and 2) the comprehensibility of the interpretation; and 3) to develop new technology and methods for analyzing and supporting print interpreting. The process means in the narrow sense the real-time conversion act and the changes in the message; in the broader sense it covers the whole communicative event, including the activity of interpreting and the actions of the participants and their interaction. The comprehensibility will be examined in terms of readability and coherence. The main research methods are textual and multimodal analysis, and eye movement analysis. Because the research problems are multidisciplinary, they will be studied in an interdisciplinary collaboration combining approaches from Linguistics, Translation Studies (especially Interpreting Studies), and Computer Sciences.

The practical aim of the study is to develop new technological solutions and to improve the accessibility of communication. Results on the reading process can provide valuable information to develop better ways of rendering the text, and thereby help in making the hearing impaired persons more equal partners in the ubiquitous communication situation. In addition, the study will contribute to deeper understanding of the relationship between writing and speaking, verbal and non-verbal communication, and produce new information of their interchangeability in various media.