Towards Visually-Guided Movie Subtitle Translation for Indic Languages

Tarun Chintada, Kshetrimayum Boynao Singh, Asif Ekbal


Abstract
Movie subtitle translation is inherently multimodal, yet text-only systems often miss visual cues needed to convey emotion, action, and social nuance, especially for low-resource Indic languages (English to Hindi, Bengali, Telugu, Tamil and Kannada). We present a case study on five full-length films and compare two lightweight visual grounding strategies: structured attribute summaries from a 5-minute sliding window and free-text summaries of inter-subtitle visual gaps. Our analysis shows that temporal misalignment between subtitles and frames is a major obstacle in long-form video, often rendering indiscriminate visual grounding ineffective. However, oracle selective grounding, which replaces only the lowest-quality 20-30
Anthology ID:
2026.eamt-2.28
Volume:
Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 2)
Month:
June
Year:
2026
Address:
Tilburg, The Netherlands
Editors:
Dimitar Shterionov, Eva Vanmassenhove, Mirella De Sisto, Fred Blain, Javad Pourmostafa Roshan Sharami, Lisa Lepp, Chiara Manna, Argentina Anna Rescigno, Alina Karakanta, Ayla Rigouts Terryn, Manuel Lardelli, Natalia Resende, Elena Murgolo, Janiça Hackenbuchner, Anna Zaretskaya, Miquel Esplà-Gomis, Thierry Etchegoyhen, Dagmar Gromann, Rachel Bawden, Barry Haddow, Sara Szoc, Mikel Forcada, Helena Moniz
Venue:
EAMT
SIG:
Publisher:
European Association for Machine Translation
Note:
Pages:
80–90
Language:
URL:
https://aclanthology.org/2026.eamt-2.28/
DOI:
Bibkey:
Cite (ACL):
Tarun Chintada, Kshetrimayum Boynao Singh, and Asif Ekbal. 2026. Towards Visually-Guided Movie Subtitle Translation for Indic Languages. In Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 2), pages 80–90, Tilburg, The Netherlands. European Association for Machine Translation.
Cite (Informal):
Towards Visually-Guided Movie Subtitle Translation for Indic Languages (Chintada et al., EAMT 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.eamt-2.28.pdf