ERC Advanced Grant Project CALCULUS: Extending the Boundary of Machine Translation

Jingyuan Sun, Mingxiao Li, Ruben Cartuyvels, Marie-Francine Moens


Abstract
The CALCULUS project, drawing on human capabilities of imagination and commonsense for natural language understanding (NLU), aims to advance machine-based NLU by integrating traditional AI concepts with contemporary machine learning techniques. It focuses on developing anticipatory event representations from both textual and visual data, connecting language structure to visual spatial organization and incorporating broad knowledge domains. Through testing these models in NLU tasks and evaluating their ability to predict untrained spatial and temporal details using real-world metrics, CALCULUS employs machine learning methods, including Bayesian techniques and neural networks, especially in data-sparse scenarios. The project’s culmination involves creating demonstrators that transform written stories into dynamic videos, showcasing the interdisciplinary expertise of the project leader in natural language processing, language and visual data analysis, information retrieval, and machine learning, all vital for the project’s achievements. In the CALCULUS project, our exploration of machine translation extends beyond the conventional text-to-text framework. We are broadening the horizons of machine translation by delving into the essence of transforming the formats of data distribution while keeping the meaning. This innovative approach involves converting information from one modality into another, transcending traditional linguistic boundaries. Our project includes novel work on translating text into images and videos, brain signals into images and videos.
Anthology ID:
2024.eamt-2.9
Volume:
Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 2)
Month:
June
Year:
2024
Address:
Sheffield, UK
Editors:
Carolina Scarton, Charlotte Prescott, Chris Bayliss, Chris Oakley, Joanna Wright, Stuart Wrigley, Xingyi Song, Edward Gow-Smith, Mikel Forcada, Helena Moniz
Venue:
EAMT
SIG:
Publisher:
European Association for Machine Translation (EAMT)
Note:
Pages:
16–17
Language:
URL:
https://aclanthology.org/2024.eamt-2.9
DOI:
Bibkey:
Cite (ACL):
Jingyuan Sun, Mingxiao Li, Ruben Cartuyvels, and Marie-Francine Moens. 2024. ERC Advanced Grant Project CALCULUS: Extending the Boundary of Machine Translation. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 2), pages 16–17, Sheffield, UK. European Association for Machine Translation (EAMT).
Cite (Informal):
ERC Advanced Grant Project CALCULUS: Extending the Boundary of Machine Translation (Sun et al., EAMT 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eamt-2.9.pdf