題目：Recent Empirical Advances in Multimodal and Multilingual Language Processing in Subtitles and Captions
地點：ZOOM ID: 750 888 0655
The use of video has accelerated rapidly in the past decade owing to technological developments, globalisation, trends in new media, and a range of other factors. We now regularly interact with video, particularly for entertainment, information, and education. Such videos, however, represent a language and accessibility barrier, especially for those who need or may benefit from support in a first or foreign language. Subtitles (interlingual) and captions (intralingual) represent a unique solution, and despite indications of their ability to enhance language processing and related performance, this potential has yet to be realised due to methodological and technological limitations and a limited evidence base leading to a divergence between academic research and industry practice.
In this talk, I will report on a series of recent collaborative projects which have explored L1 and L2 language processing in subtitles and captions within educational and entertainment settings. I will describe how methodologies have developed to move from self-reports, psychometrics and task performance, including comprehension and recall, to online methods of eye tracking and electroencephalography with a focus on cognitive load and immersion. Through these methodological innovations, we have examined various aspects of the complex multimodal and multilingual language processing, including the impact of errors and discreet linguistic targets, ‘cognitively efficient’ captions, viewers’ integration of bimodal input, and comparisons of monolingual and bilingual subtitles, and machine- and human-generated subtitles and captions.
In closing, I will reflect on the abovementioned methods and situate the reported findings within a broader context of on-going developments in language technologies. I also show how subtitles and captions represent an attractive and stimulating testing ground for future investigations of language processing in multilingual and multimodal contexts leading to refinement of evidence-based models and large-scale local and global applications.
ZOOM 鏈接： https://zoom.com.cn/j/7508880655