Description of research and teaching

I do research on multimodality, that is, how multiple modes of expression interact and co-operate with each other in different communicative situations.

To exemplify, spoken language, gestures, posture and gaze are constantly coordinated in face-to-face interaction, while magazines, newspapers, websites and other page-based texts organize written language, photographs, illustrations, diagrams, information graphics and other modes of expression into coherent layouts.

In both cases, communication builds on appropriate combinations of different modes of expression. Theories of multimodality attempt to explain how such appropriate combinations are formed and how they become understandable in context.

I am particularly interested in how theories of multimodality can inform research on artificial intelligence, and conversely, how artificial intelligence can support empirical research on multimodality. In addition, I am also interested in applications of natural language processing and computer vision in the humanities and beyond.

Education information

• Title of Docent in Multimodality Research and Digital Methods, University of Turku, Finland (2022)
  
• Doctor of Philosophy in English Philology from the University of Helsinki, Finland (2014)
• Master of Arts in English Philology from the University of Helsinki (2007)
• Bachelor of Arts in English Philology, with minor subjects in Communication, General Linguistics and Russian and East European Studies from the University of Helsinki (2005)