What is top P (nucleus sampling) in AI chat?
Updated
Top P, also called nucleus sampling, is a sampler setting that limits a model's choice of next token to the smallest group of most likely tokens whose probabilities add up to P. At top P 0.9 the model ranks the candidates, keeps them until they cover 90% of the probability, throws away the long tail of unlikely words, and picks from what is left. At 1 it is off. UnsavedAI's API accepts values above 0 up to 1.
Try it now, free, no account neededStart a story with LokiThe trickster god needs an accomplice. You'll do.A worked example
Suppose the story so far is "Loki ___ at your question." and the model scores the next word like this:
- grins: 45%
- smirks: 25%
- laughs: 15%
- sighs: 8%
- hisses: 4%
- thousands of other words: 3% between them
Top P adds the probabilities from the top down until it reaches P:
- Top P 1: nothing is removed; even the 3% tail can be picked.
- Top P 0.9: grins, smirks and laughs reach 85%, so sighs is added to pass 90%. Four words survive.
- Top P 0.7: grins and smirks reach 70%. Two words survive.
The survivors are then rescaled to add up to 100% again (at 0.9, grins becomes about 48%) and one is drawn at random. With Loki, the 0.9 setting keeps his reactions varied; 0.7 makes him grin or smirk every single time.
Why it is called nucleus sampling
The name comes from a 2019 research paper that described the likely words as the "nucleus" of the distribution. It was proposed to fix two opposite failures: always picking the single top word makes text dull and loopy, while sampling from every word lets strange choices from the tail slip in. Cutting the tail by total probability, rather than by a fixed count, keeps the variety and drops most of the junk.
How it behaves with other samplers
Because the cut follows the running total, top P adapts to the model's confidence. When one word has 95% on its own, only that word survives. When the model is unsure and probability is spread thin, dozens survive. That differs from top K, which keeps a fixed number of words, and from min P, which keeps words above a share of the top word's probability.
Temperature interacts with it. A higher temperature flattens the probabilities, so more words are needed to reach 0.9 and more of the tail gets in. Whether temperature is applied before or after the top P cut depends on the backend's sampler order, which is why the same pair of numbers can feel different on two services.
Common mistakes
- Very low values, like 0.5, to make a character "focused". The result is the same phrasing on every swipe.
- Typing 0 to turn it off. Off is 1; on UnsavedAI, 0 is rejected with an error naming top_p.
- Stacking a tight top P with a tight top K and a high min P. Each filter removes more, and replies go flat.
- Expecting it to fix logic or memory. It only changes which words are eligible.
- Mixing it up with top A or typical P, separate samplers SillyTavern lists for some backends.
Where you set it
In SillyTavern it is the Top P slider in AI Response Configuration; labels can change between versions. In an OpenAI-style request it is a number field, "top_p": 0.95. Local backends such as KoboldCpp have it too. The UnsavedAI website picks samplers for you; with an API key, the value you send is what the model gets. For the values to start from and what to change for each symptom, see roleplay sampler settings.
Questions people ask
Should I use top P or min P?
Either can be your main filter. Min P adapts to the top word's strength, top P to the running total; using both at gentle values is fine, tight values on both is not.
What does top P 1 mean?
Off. Every word stays eligible and only temperature and other samplers shape the choice.
Is top P the same as top K?
No. Top K keeps a fixed number of words; top P keeps however many it takes to reach a share of the probability.
