How to choose the right AI model for customer support
Choosing the right AI model for customer support is a matching exercise, not a search for the strongest model. Match the model to the task (a customer draft needs more than a preview summary), to the mailbox (a legal inbox has different rules from a general one), and to your data policy. Then confirm the match on real tickets. ReplyRabbit is built for this: provider and model are set per mailbox and per task.
Step one: decide by task
ReplyRabbit runs several kinds of AI work, and they don't all deserve the same model.
| Task | What it needs | Sensible choice |
|---|---|---|
| Draft replies | Judgement, tone, restraint | Your strongest affordable model |
| Next Actions and AI Signals | Classification and short lists | Usually the draft model; they share the Drafts setting |
| Embeddings (semantic search) | An embeddings endpoint | A provider that offers one; Anthropic doesn't |
| Mailbox previews (Team) | Short summaries, high volume | The cheap default |
| Attachment vision (Team) | Reading screenshots and PDFs | The cheap default, or a vision-capable model you've tested |
The AI Provider section has a subsection for each, and blank key fields reuse the key from Drafts. See Per-task AI models.
Step two: decide by mailbox
Because every mailbox has its own provider, you can answer the privacy question per inbox rather than for the whole company.
- General support: a cloud provider, chosen for draft quality.
- Billing, legal, medical, or anything under a strict data agreement: a local Ollama or LM Studio model on the Pro plan, so the email never leaves the server.
- High-volume, low-stakes inbox: a fast, low-cost cloud model, with Auto Draft on.
- A test mailbox: OpenRouter, so you can switch model IDs without new accounts.
Step three: decide by constraint
Some choices are made for you.
- On the free plan, the provider is OpenAI. Every other provider needs Pro or Team.
- If no customer data may leave the server, only local models qualify.
- If you use semantic search, the embeddings provider must offer embeddings, so an Anthropic-only setup needs a second provider for that task.
- If the FreeScout host can't make outbound requests to a vendor, that vendor is out.
Step four: confirm on real tickets
A shortlist isn't a decision. Put the candidate on a mailbox with company context and response style filled in, turn on Auto Draft, and let agents work for a week. Measure how often drafts go out with light edits, how often they're rewritten, and how often they contain something invented. Then look at the provider bill for the same week.
If the candidate wins on edits but loses badly on cost, try it as the draft model with a cheaper model on previews, rather than dropping it.
A worked example
A small software company runs three FreeScout mailboxes. Support goes on Anthropic for draft quality, with OpenAI in the Embeddings subsection because Claude has no embeddings endpoint. Billing goes on Ollama, because invoices and card details appear in those threads. The partner inbox, which mostly gets forwarded newsletters, stays on the free-plan default with Auto Draft off, since most of it is filtered as noise anyway.
None of that required a different architecture, only different mailbox settings. When one choice turns out wrong, it's a dropdown change.
Where to go next
- Compare the providers themselves in Choosing an AI provider.
- Set up the mailbox fields that make any model sound like you in How to set up ReplyRabbit.
- If drafts fail or stop, check Troubleshooting ReplyRabbit and the FAQ.
Official references
Provider names, model lists, pricing, privacy terms, and legal guidance change. Check the current source before setting a production policy.