How to use SillyTavern without a GPU
Updated
SillyTavern itself never needs a GPU; only running a model on your own machine does. Connect SillyTavern to a hosted OpenAI-compatible API instead and the model runs on the provider's servers. With UnsavedAI, choose Chat Completion, source Custom (OpenAI-compatible), paste the base URL ending in /v1 and your key, and pick gemma-4-12b, free on every plan.
Try it now, free, no account neededStart a story with The NarratorAny story you want, you chooseWhy people think they need a GPU
A lot of SillyTavern advice comes from people who run models locally, where the graphics card's memory decides which models fit and how fast they write. That is a real constraint for local setups, but it belongs to the model backend, not to SillyTavern. SillyTavern is a lightweight app that formats prompts and shows replies; an ordinary laptop runs it comfortably.
Local model or hosted model
- Local: free per message and fully offline, but limited by your hardware, often slow on smaller cards, and you maintain the backend yourself.
- Hosted: works on any machine, larger models than most home cards can hold, and nothing to keep updated. In exchange your prompts travel to the provider, so check what they keep.
UnsavedAI never stores API traffic, and nobody reads or screens it. Your chat history stays in SillyTavern on your own machine either way.
The settings to use
Open API Connections with the plug icon. Set API to Chat Completion and Chat Completion Source to Custom (OpenAI-compatible). Paste our base URL into Custom Endpoint (Base URL) and a key made for SillyTavern into Custom API Key, click Connect, and pick gemma-4-12b. Priority members can pick gemma-4-31b.
In AI Response Configuration set Context Size to 56,000 and Max Response Length to 700 (800 for gemma-4-31b). Temperature 0.9 and Top P 0.95 are a good start. The full walkthrough, with screenshots of where each field lives, is in the SillyTavern guide.
What you get on each plan
Free gives you gemma-4-12b with a 61,440-token context and no daily token cap. It writes one reply at a time and runs on spare capacity, so at busy times you wait in line. Priority, $4.99 a month, adds gemma-4-31b, writes two replies at once and goes ahead of waiting Free requests. The Free vs Priority comparison covers when the upgrade is worth it.
Keeping it quick
Hosted models reuse the start of your prompt between messages, so the next reply only has to read what is new. Anything near the top that changes every turn, such as a {{time}} or {{random}} macro in the system prompt, forces a full re-read and adds seconds before the first word. Keep those out of the card, and see context size for roleplay for the rest.
Questions people ask
Can I run SillyTavern on a phone?
SillyTavern is built to run on a computer or a small server; some people run it on Android through a terminal app. Either way a hosted model does the heavy work, so the device only needs to run the interface.
Is a hosted 12B better than a local 7B?
Usually a bigger model follows character cards more closely, but taste varies. Try the same character on both for twenty messages and keep the one you prefer.
Will my chats leave my computer?
Each prompt is sent to the provider to get a reply. SillyTavern still keeps the history locally, and UnsavedAI does not store API traffic.
