Best-First Beam Search

Clara Meister, Tim Vieira, Ryan Cotterell


Abstract
Decoding for many NLP tasks requires an effective heuristic algorithm for approximating exact search because the problem of searching the full output space is often intractable, or impractical in many settings. The default algorithm for this job is beam search—a pruned version of breadth-first search. Quite surprisingly, beam search often returns better results than exact inference due to beneficial search bias for NLP tasks. In this work, we show that the standard implementation of beam search can be made up to 10x faster in practice. Our method assumes that the scoring function is monotonic in the sequence length, which allows us to safely prune hypotheses that cannot be in the final set of hypotheses early on. We devise effective monotonic approximations to popular nonmonontic scoring functions, including length normalization and mutual information decoding. Lastly, we propose a memory-reduced variant of best-first beam search, which has a similar beneficial search bias in terms of downstream performance, but runs in a fraction of the time.
Anthology ID:
2020.tacl-1.51
Volume:
Transactions of the Association for Computational Linguistics, Volume 8
Month:
Year:
2020
Address:
Cambridge, MA
Editors:
Mark Johnson, Brian Roark, Ani Nenkova
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
795–809
Language:
URL:
https://aclanthology.org/2020.tacl-1.51
DOI:
10.1162/tacl_a_00346
Bibkey:
Cite (ACL):
Clara Meister, Tim Vieira, and Ryan Cotterell. 2020. Best-First Beam Search. Transactions of the Association for Computational Linguistics, 8:795–809.
Cite (Informal):
Best-First Beam Search (Meister et al., TACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.tacl-1.51.pdf