Sample Strategies Overview

Sample strategies are techniques used to generate text from a model. They determine how the model selects the next token during generation, balancing between randomness and coherence.

Common Strategies

  • Greedy Search: Selects the most probable token at each step.
  • Beam Search: Explores multiple sequences simultaneously for better results.
  • Top-k Sampling: Limits the selection to the top-k most probable tokens.
  • Nucleus Sampling: Chooses from a dynamic set of top tokens based on cumulative probability.