Choosing an AI provider
ReplyRabbit works with seven AI providers. Choose based on three things: cost, the quality of the drafts you want, and whether your data has to stay on your own hardware. If you are unsure, start with OpenAI or Anthropic and change it later in your mailbox settings.
The seven providers
Five providers are cloud services and two run locally on your own machine. Cloud providers bill per token (you pay them directly); local providers have no per-token cost.
| Provider | Type | Best for | Pricing |
|---|---|---|---|
| OpenAI | Cloud | The usual starting point. Available on the Free tier, so you can try ReplyRabbit with one provider before upgrading. | platform.openai.com |
| Anthropic | Cloud | Claude models. A strong all-round choice for natural, careful support replies. | anthropic.com |
| Google AI Studio | Cloud | Gemini models, including fast, low-cost options. | ai.google.dev |
| OpenRouter | Cloud | One API key, many models from different vendors. Good for experimenting and comparing models without signing up everywhere. | openrouter.ai |
| Z.AI | Cloud | GLM models. A capable, cost-conscious cloud option. | z.ai |
| Ollama (Local) | Local | Runs models on your own hardware (localhost:11434), so nothing leaves your server and there is no per-token cost. | ollama.com |
| LM Studio (Local) | Local | Runs models on your own hardware (localhost:1234), private with no per-token cost. | lmstudio.ai |
Free tier uses OpenAI
Cloud vs local
Cloud providers (OpenAI, Anthropic, Google AI Studio, OpenRouter, Z.AI) are the easiest way to start and usually offer the top draft quality. You pay per token, and the conversation content used for enabled AI features, such as drafts, next actions, and AI Signals, leaves your server and is sent to that provider's API.
Local providers (Ollama, LM Studio) run the model on your own hardware. Nothing leaves your server, and there is no per-token cost. The trade-off is that you need a reasonably capable machine, and draft quality depends on the local model you run. Local models require a paid tier (Pro or Team).
Prompt caching and token budgets
ReplyRabbit handles provider prompt caching automatically when it is safe for the selected provider. There is no mailbox setting to turn on.
- OpenAI: ReplyRabbit sends a safe cache routing key for direct OpenAI requests so repeated support prompts can be routed consistently.
- Anthropic: ReplyRabbit uses Claude prompt caching only when it sees likely reuse across matching draft requests. If Anthropic rejects cache control, ReplyRabbit retries the normal prompt.
- Gemini, OpenRouter, and Z.AI: ReplyRabbit records cache token metadata when the provider returns it, but it does not force a generic cache setting through those routes.
- Ollama and LM Studio: local models do not use provider prompt caching. They can still avoid per-token cloud billing because generation runs on your hardware.
Monthly Token Budget remains conservative. ReplyRabbit counts total input and output tokens, even when a provider bills cached input differently, because cached prompt tokens still use context and may count toward provider limits.
Which should I choose?
- Just getting started: use OpenAI or Anthropic. They give good results out of the box, and OpenAI is available on the Free tier so you can try ReplyRabbit first.
- Want to compare models: use OpenRouter. One API key gives you access to many models from different vendors, so you can experiment without separate accounts.
- Privacy or cost control matters most: use Ollama or LM Studio. Your customer data never leaves your server and there is no per-token bill, at the cost of running the model on your own hardware (Pro or Team tier).
Next