Exploring multimodal strategies: Understanding sign language interpretation in remote educational contexts

Application summary

This study investigates the interpreting process of Finnish – Finnish Sign Language (FinSL) interpreters in online educational settings, focusing on their strategic choices and multimodal interpreting strategies. Additionally, by using eye-tracking technology, the study examines how these strategies interact with the visual attention of deaf students, particularly how they shift their gaze between slides and the interpreter when following an interpreted online lecture. This project explores how interpreters integrate multiple semiotic resources – such as gesture, gaze, and artefacts – into multimodal wholes. It seeks to uncover how these multimodal interpreting strategies interact with deaf students’ gaze patterns, providing a deeper understanding of communication in remote educational settings and offering insights for enhancing interpreting practices. The research comprises three sub-studies: (1) analysing the multimodal interpreting strategies used by interpreters in remote educational settings; (2) investigating the motivations behind interpreters’ strategic choices through retrospective interviews; and (3) examining how these strategies guide the visual attention of deaf students using eye-tracking technology. These sub-studies will draw on datasets from twelve FinSL interpreters and ten deaf students. The project employs interdisciplinary methods, incorporating cognitive science tools such as eye-tracking, and addresses global trends towards online learning environments. Through its multimodal approach, the study aims to advance theoretical concepts in interpreting studies, improve educational interpreting practices, and contribute to more inclusive and equitable learning environments.