Yotam Intrator
2024
Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications?
Yotam Intrator
|
Matan Halfon
|
Roman Goldenberg
|
Reut Tsarfaty
|
Matan Eyal
|
Ehud Rivlin
|
Yossi Matias
|
Natalia Aizenberg
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)
Large language models hold significant promise in multilingual applications. However, inherent biases stemming from predominantly English-centric pre-training have led to the widespread practice of pre-translation, i.e., translating non-English inputs to English before inference, leading to complexity and information loss. This study re-evaluates the need for pre-translation in the context of PaLM2 models, which have been established as highly performant in multilingual tasks. We offer a comprehensive investigation across 108 languages and 6 diverse benchmarks, including open-end generative tasks, which were excluded from previous similar studies. Our findings challenge the pre-translation paradigm established in prior research, highlighting the advantages of direct inference in PaLM2. Specifically, PaLM2-L consistently outperforms pre-translation in 94 out of 108 languages. These findings pave the way for more efficient and effective multilingual applications, alleviating the limitations associated with pre-translation and unlocking linguistic authenticity.
Streamlining Conformal Information Retrieval via Score Refinement
Yotam Intrator
|
Regev Cohen
|
Ori Kelner
|
Roman Goldenberg
|
Ehud Rivlin
|
Daniel Freedman
Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
Information retrieval (IR) methods, like retrieval augmented generation, are fundamental to modern applications but often lack statistical guarantees. Conformal prediction addresses this by retrieving sets guaranteed to include relevant information, yet existing approaches produce large-sized sets, incurring high computational costs and slow response times. In this work, we introduce a score refinement method that applies a simple monotone transformation to retrieval scores, leading to significantly smaller conformal sets while maintaining their statistical guarantees. Experiments on various BEIR benchmarks validate the effectiveness of our approach in producing compact sets containing relevant information.
Search
Co-authors
- Roman Goldenberg 2
- Ehud Rivlin 2
- Matan Halfon 1
- Reut Tsarfaty 1
- Matan Eyal 1
- show all...