Gaussian Process Optimization for Adaptable Multi-Objective Text Generation using Linearly-Weighted Language Models

Mohammad Mahdi Abdollah Pour, Ali Pesaranghader, Eldan Cohen, Scott Sanner


Abstract
In multi-objective text generation, we aim to optimize over multiple weighted aspects (e.g., toxicity, semantic preservation, fluency) of the generated text. However, multi-objective weighting schemes may change dynamically in practice according to deployment requirements, evolving business needs, personalization requirements on edge devices, or the availability of new language models and/or objective requirements. Ideally, we need an efficient method to adapt to the dynamic requirements of the overall objective. To address these requirements, we propose a linear combination of objective-specific language models to efficiently adapt the decoding process and optimize for the desired objective without the significant computational overhead of retraining one or more language models. We show empirically that we can leverage Gaussian Process black box optimization to adapt the language model decoder weights to outperform other fixed weighting schemes and standard baselines of the task in only a few iterations of decoding. Overall this approach enables highly efficient adaptation of controllable language models via multi-objective weighting schemes that may evolve dynamically in practical deployment situations.
Anthology ID:
2024.findings-naacl.99
Volume:
Findings of the Association for Computational Linguistics: NAACL 2024
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1529–1536
Language:
URL:
https://aclanthology.org/2024.findings-naacl.99
DOI:
Bibkey:
Cite (ACL):
Mohammad Mahdi Abdollah Pour, Ali Pesaranghader, Eldan Cohen, and Scott Sanner. 2024. Gaussian Process Optimization for Adaptable Multi-Objective Text Generation using Linearly-Weighted Language Models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1529–1536, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
Gaussian Process Optimization for Adaptable Multi-Objective Text Generation using Linearly-Weighted Language Models (Abdollah Pour et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-naacl.99.pdf
Copyright:
 2024.findings-naacl.99.copyright.pdf