Best JanitorAI generation settings for a proxy model
Updated
With UnsavedAI as your JanitorAI proxy, set Context size to 56,000 (both Gemma models allow 61,440 including the reply), Max new tokens to about 700 for gemma-4-12b or 800 for gemma-4-31b, and Temperature to 0.9. Those three settings decide whether long chats keep working, whether replies finish, and how lively the character sounds.
Try it now, free, no account neededStart a story with The NarratorAny story you want, you chooseContext size: 56,000
Context size is how much of the chat JanitorAI packs into each request: the character card, your persona, the history and your new message. Our Gemma models allow 61,440 tokens in total, and that total includes the reply. If JanitorAI sends a prompt that leaves no room to answer, the request fails with "Prompt must leave room for output within the ...-token context."
56,000 leaves room for a full reply plus a safety margin, because JanitorAI counts tokens its own way and not exactly as the model does. When the chat grows past that, JanitorAI quietly drops the oldest messages instead of failing. Setting it higher does not give you more memory; it only makes long chats break once they fill up.
Max new tokens: 700 to 800, or 0
This caps how long each reply can be. Around 700 for gemma-4-12b and 800 for gemma-4-31b gives a few solid paragraphs without the model rambling. If you prefer short, punchy back-and-forth, 300 to 400 works, but expect the occasional reply that stops mid-sentence when the character has more to say.
You can also set 0, which tells us to use our default. Whatever you choose, it has to be a whole number; a decimal or a blank value returns "Output token limit must be a positive integer."
Temperature: 0.9
Temperature controls how adventurous word choice is. 0.9 suits most characters on Gemma: varied, warm, still on track. Drop to 0.8 for a mystery or a character who must stay precise and consistent. Go up to 1.0 for chaotic comedy or a narrator who should surprise you. Past about 1.2 most characters start drifting and contradicting themselves.
If you are copying a preset made for another model, check temperature first. Values tuned for very different models are the usual cause of a character that suddenly sounds wrong.
What the other fields do
Settings JanitorAI sends that we recognise, such as top_p or a repetition penalty, are passed to the model if they are in range. Fields we do not use are ignored rather than rejected, so an unusual preset will not break the connection. If one value is out of range, the error names it, for example "temperature must be a number of at least 0."
For what each sampler does and when to touch it, see roleplay sampler settings.
Checking it works
After changing settings, refresh JanitorAI, open a chat with plenty of history and send a normal message. A reply that starts within a few seconds and ends on a full sentence means you are set. If it cuts off, raise Max new tokens; if it fails on long chats only, lower Context size. The JanitorAI guide lists every error message and its fix.
Questions people ask
Should I use 61,440 as the context size?
No. That limit includes the reply, so a prompt that large leaves no room to answer. 56,000 or less keeps long chats working.
Is the context size different on Priority?
No. gemma-4-12b and gemma-4-31b both allow 61,440 tokens on either plan, so 56,000 is right for both.
Why are my replies so short even with a high limit?
Max new tokens is a ceiling, not a target. Short replies usually come from the character card or your own short messages; write longer turns and add a longer example message to the card.
