Beam Search

In the context of natural language processing and sequence generation tasks, what is the beam search algorithm, and how does it differ from greedy decoding? Explain the core idea behind beam search, including how it explores the search space, maintains a set of candidate sequences, and makes decisions about the most likely output sequence. Discuss the trade-offs involved in selecting the beam width parameter and how beam search addresses issues like diversity versus accuracy in generating sequences. Furthermore, can you highlight any limitations or scenarios where beam search might produce suboptimal results?